Models and experts: urgent questions about how we inform decisions and public policy

Curated on Posted on by Stefaan Verhulst

Blog and book by Erica Thompson: “Mathematical models are here to stay. Whether they are determining supply chain vulnerabilities, demonstrating regulatory compliance, or informing policies for a zero-carbon future, quantitative models are at the heart of modern societies. And as computers become more powerful and more readily accessible, artificial intelligence and machine learning models are also being applied in many new areas.

Given that, we urgently need to understand how best to use and work with models to make good and responsible decisions. Statistician George Box was quite right to point out that “all models are wrong”. They are necessarily simplifications of the messy reality we want to get to grips with. But many quantitative methods for working with models basically assume that the model is right, or at least that it can accurately estimate the range of plausible outcomes.

If the model is not quite perfect, we can expect some of its outputs to be wrong (not just inaccurate). In that case, the information that is offered as decision support could be misleading. We have two options here. We could remain in what I call model land and just expect to have to say “what a shame, we made the wrong decision” occasionally. In some circumstances that might be a reasonable answer, but if we are making decisions about critical infrastructure or selling a product that might be unsafe to millions of people, then we have both a legal and ethical responsibility to do better, to get out of model land and understand how relevant our model results are for the real world.

So what’s the second option? You won’t be surprised to know that it isn’t easy. In my new book, I consider some of the implications of working with imperfect models and the kinds of strategies that we need to adopt to make best use of the information they contain. One theme that I explore is the need to understand the role of expert judgement in constructing, calibrating, evaluating, and using models, and the way that that expert judgement might be shaped by our social context.

Experts make models – and that’s a very good thing, because who would want to rely on a model created by a non-expert? But their expertise is often limited, and it comes from a particular background and set of experiences. Indeed, you can often find equally qualified experts who will disagree about the right assumptions to make when constructing a model and who give different advice about how to achieve the stated aims. Then the decision-maker – probably a non-expert – will be in the difficult position of trying to adjudicate between different models from different experts, weighing up their relative credibility…(More)”.

Mapping Diversity

Curated on Posted on by Stefaan Verhulst

About: “Mapping Diversity is a platform for discovering key facts about diversity and representation in street names across Europe, and to spark a debate about who is missing from our urban spaces.

We looked at the names of 145,933 streets across 30 major European cities, located in 17 different countries. More than 90% of the streets named after individuals are dedicated to white men. Where did all the other inhabitants of Europe end up? The lack of diversity in toponymy speaks volumes about our past and contributes to shaping Europe’s present and future…(More)”.

Principles for effective beneficial ownership disclosure

Curated on Posted on by Stefaan Verhulst

Open Ownership: “The Open Ownership Principles (OO Principles) are a framework for considering the elements that influence whether the implementation of reforms to improve the transparency of the beneficial ownership of corporate vehicles will lead to effective beneficial ownership disclosure, that is, it generates high-quality and reliable data, maximising usability for users.

The OO Principles are intended to support governments implementing effective beneficial ownership transparency reforms and guide international institutions, civil society, and private sector actors in understanding and supporting reforms. They are a tool to identify and separate issues affecting implementation, and they provide a framework for assessing and improving existing disclosure regimes. If implemented together, the OO Principles enable disclosure systems to generate actionable and usable data across the widest range of policy applications of beneficial ownership data.

The nine principles are interdependent, but can be broadly grouped by the three main ways they improve data. The Definition, Coverage, and Detail principles enable data disclosure and collection. The Central register, Access, and Structured data principles facilitate data storage and auditability. Finally, the Verification, Up-to-date and historical records, and Sanctions and enforcement principles improve data quality and reliability….Download January 2023 version (translated versions are forthcoming)”

Decidim: why digital tools for democracy need to be developed democratically

Curated on Posted on by Stefaan Verhulst

Blog by Adrian Smith and Pedro Prieto Martín: “On Wednesday 18 January 2023, a pan-European citizen jury voted Barcelona the first European Capital of Democracy. Barcelona has a rich history of official and citizen initiatives in political and economic democracy. One received a special mention from the jurors. That initiative is Decidim.

Decidim is a digital platform for citizen participation. Through it, citizens can propose, comment, debate, and vote on urban developments, decide how to spend city budgets, and design and contribute to local strategies and plans.

Launched in 2016, more than 400 organisations around the world have since used the platform. What makes Decidim stand out, according to our research, is developer commitment to democratising technology development itself and embedding it within struggles for democracy offline and online. Decidim holds important lessons at a time when the monopolisation of social media by corporate power presents democrats with so many challenges…(More)”.

The Sensitive Politics Of Information For Digital States

Curated on Posted on by Stefaan Verhulst

Essay by Federica Carugati, Cyanne E. Loyle and Jessica Steinberg: “In 2020, Vice revealed that the U.S. military had signed a contract with Babel Street, a Virginia-based company that created a product called Locate X, which collects location data from users across a variety of digital applications. Some of these apps are seemingly innocuous: one for following storms, a Muslim dating app and a level for DIY home repair. Less innocuously, these reports indicate that the U.S. government is outsourcing some of its counterterrorism and counterinsurgency information-gathering activities to a private company.

While states have always collected information about citizens and their activities, advances in digital technologies — including new kinds of data and infrastructure — have fundamentally altered their ability to access, gather and analyze information. Bargaining with and relying on non-state actors like private companies creates tradeoffs between a state’s effectiveness and legitimacy. Those tradeoffs might be unacceptable to citizens, undermining our very understanding of what states do and how we should interact with them …(More)”

Whole of government innovation

Curated on Posted on by Stefaan Verhulst

Report by Geoff Mulgan: ‘Whole of government’ approaches – that aim to mobilise and align many ministries and agencies around a common challenge – have a long history. There have been notable examples during major wars, and around attempts to digitize societies, to cut energy use and to respond to the COVID-19 pandemic.

This paper has been prepared as part of a European Commission programme which I’m chairing looking at ‘whole of government innovation’ and working with national governments to help them better align their actions.

My paper – linked below – looks at the lessons of history. It outlines the many tools governments can use to achieve cross-cutting goals, linking R&D to law, regulation and procurement, and collaborating with business, universities and civil society. It argues that it is unwise to rely only on committees and boards. It shows how these choices link to innovation strategy and funding, including the relevance of half a century of experiment with moon-shots and missions.

The paper describes how the organisational challenges vary depending on the nature of the task; why governments need to avoid common technology or ‘STI trap’, of focusing only on hardware and not on social arrangements or business models; why constellations and flotillas of coordination are usually more realistic than true ‘whole of government approaches; the importance of mobilising hearts and minds as well as money and command.

Finally, it addresses the relevance of different approaches to current tasks such as the achievement of a net zero economy and society. The paper is shared as a working document – I’m keen to find new examples and approaches…(More)”.

Citizen Z: Strengthening the participation of young citizens in democratic and civic life

Curated on Posted on by Stefaan Verhulst

Blog of the European Commission: “Contemporary political attitudes are characterized by significant political indifference, disengagement from public life, and a decline in political participation, especially among young people, whose level of interest in politics is steadily declining in almost all EU countries. The ‘Citizen Z’ project, which started in November 2022, applies deliberative methods, to both stimulate interest in civic and political life among young people aged 15 to 25 and to involve them in the decision-making process. Special attention is devoted to the intersectionality principle and those groups that are often affected by low levels of participation: migrants, disadvantaged communities, girls and women.

The ‘Citizen Z’ project aims to respond to the European Commission priorities outlined in the European Democracy Action Plan and the EU Citizenship Report 2020 by enhancing civic engagement and democratic participation of EU citizens, particularly youth aged 15 to 25.

The project is also in line with the position supported by the European Committee of the Regions, which in 2019 encouraged a ‘cultural change’ towards experimenting with deliberative democracy tools as developed at the local level (European Committee of the Regions, Putting citizens at the Centre of the EU agenda), as the most authentic democratic participation originates in the context closest to the citizen…(More)”.

Democracy, Agony, and Rupture: A Critique of Climate Citizens’ Assemblies

Curated on Posted on by Stefaan Verhulst

Paper by Amanda Machin: “Stymied by preoccupation with short-term interests of individualist consumers, democratic institutions seem unable to generate sustained political commitment for tackling climate change. The citizens’ assembly (CA) is promoted as an important tool in combatting this “democratic myopia.” The aim of a CA is to bring together a representative group of citizens and experts from diverse backgrounds to exchange their different insights and perspectives on a complex issue. By providing the opportunity for inclusive democratic deliberation, the CA is expected to educate citizens, stimulate awareness of complex issues, and produce enlightened and legitimate policy recommendations. However, critical voices warn about the simplified and celebratory commentary surrounding the CA. Informed by agonistic and radical democratic theory, this paper elaborates on a particular concern, which is the orientation toward consensus in the CA. The paper points to the importance of disagreement in the form of both agony (from inside) and rupture (from outside) that, it is argued, is crucial for a democratic, engaging, passionate, creative, and representative sustainability politics…(More)”.

Democracy Report 2023: Defiance in the Face of Autocratization

Curated on Posted on by Stefaan Verhulst

New report by Varieties of Democracy (V-Dem): “.. the largest global dataset on democracy with over 31 million data points for 202 countries from 1789 to 2022. Involving almost 4,000 scholars and other country experts, V-Dem measures hundreds of different attributes of democracy. V-Dem enables new ways to study the nature, causes, and consequences of democracy embracing its multiple meanings. THE FIRST SECTION of the report shows global levels of democ- racy sliding back and advances made over the past 35 years diminishing. Most of the drastic changes have taken place within the last ten years, while there are large regional variations in relation to the levels of democracy people experience. The second section offers analyses on the geographies and population sizes of democratizing and autocratizing countries. In the third section we focus on the countries undergoing autocratization, and on the indicators deteriorating the most, including in relation to media censorship, repression of civil society organizations, and academic freedom. While disinformation, polarization, and autocratization reinforce each other, democracies reduce the spread of disinformation. This is a sign of hope, of better times ahead. And this is precisely the message carried forward in the fourth section, where we switch our focus to examples of countries that managed to push back and where democracy resurfaces again. Scattered over the world, these success stories share common elements that may bear implications for international democracy support and protection efforts. The final section of this year’s report offers a new perspective on shifting global balances of economic and trade power as a result of autocratization…(More)”.

Suspicion Machines

Curated on Posted on by Stefaan Verhulst

Lighthouse Reports: “Governments all over the world are experimenting with predictive algorithms in ways that are largely invisible to the public. What limited reporting there has been on this topic has largely focused on predictive policing and risk assessments in criminal justice systems. But there is an area where even more far-reaching experiments are underway on vulnerable populations with almost no scrutiny.

Fraud detection systems are widely deployed in welfare states ranging from complex machine learning models to crude spreadsheets. The scores they generate have potentially life-changing consequences for millions of people. Until now, public authorities have typically resisted calls for transparency, either by claiming that disclosure would increase the risk of fraud or to protect proprietary technology.

The sales pitch for these systems promises that they will recover millions of euros defrauded from the public purse. And the caricature of the benefit cheat is a modern take on the classic trope of the undeserving poor and much of the public debate in Europe — which has the most generous welfare states — is intensely politically charged.

The true extent of welfare fraud is routinely exaggerated by consulting firms, who are often the algorithm vendors, talking it up to near 5 percent of benefits spending while some national auditors’ offices estimate it at between 0.2 and 0.4 of spending. Distinguishing between honest mistakes and deliberate fraud in complex public systems is messy and hard.

When opaque technologies are deployed in search of political scapegoats the potential for harm among some of the poorest and most marginalised communities is significant.

Hundreds of thousands of people are being scored by these systems based on data mining operations where there has been scant public consultation. The consequences of being flagged by the “suspicion machine” can be drastic, with fraud controllers empowered to turn the lives of suspects inside out…(More)”.