Game Changing Tools for Evidence Synthesis: Generative AI, Data and Policy Design

Curated on Posted on by Stefaan Verhulst

Paper by Geoff Mulgan, and Sarah O’Meara: “Evidence synthesis aims to make sense of huge bodies of evidence from around the world and make it available for busy decision-makers. Google search was in some respects a game changer in that you could quickly find out what was happening in a field – but it turned out to be much less useful for judging which evidence was relevant, reliable or high quality. Now large language models (LLM) and generative AI are offering an alternative to Google and again appear to have the potential to dramatically improve evidence synthesis, in an instant bringing together large bodies of knowledge and making it available to policy-makers, members of parliament or indeed the public. 

But again there’s a gap between the promise and the results. ChatGPT is wonderful for producing a rough first draft: but its inputs are often out of date, it can’t distinguish good from bad evidence and its outputs are sometimes made up.  So nearly a year after the arrival of ChatGPT we have been investigating how generative AI can be used most effectively, and, linked to that, how new methods can embed evidence into the daily work of governments and provide ways to see if the best available evidence is being used.

We think that these will be game-changers: transforming the everyday life of policy-makers, and making it much easier to mobilise, and assess evidence – especially if human and machine intelligence are combined rather than being seen as alternatives. But they need to be used with care and judgement rather than being panaceas. [Watch IPPO’s recent discussion here]…(More)”.

Missing Persons: The Case of National AI Strategies

Curated on Posted on by Stefaan Verhulst

Article by Susan Ariel Aaronson and Adam Zable: “Policy makers should inform, consult and involve citizens as part of their efforts to data-driven technologies such as artificial intelligence (AI). Although many users rely on AI systems, they do not understand how these systems use their data to make predictions and recommendations that can affect their daily lives. Over time, if they see their data being misused, users may learn to distrust both the systems and how policy makers regulate them. This paper examines whether officials informed and consulted their citizens as they developed a key aspect of AI policy — national AI strategies. Building on a data set of 68 countries and the European Union, the authors used qualitative methods to examine whether, how and when governments engaged with their citizens on their AI strategies and whether they were responsive to public comment, concluding that policy makers are missing an opportunity to build trust in AI by not using this process to involve a broader cross-section of their constituents…(More)”.

Open Society Barometer: Can Democracy Deliver?

Curated on Posted on by Stefaan Verhulst

Open Society Foundation Report: “Between May and July of 2023, the Open Society Foundations commissioned a poll of more than 36,000 respondents from 30 countries to gauge the attitudes, concerns, and hopes of people in states with a collective population of over 5.5 billion—making it one of the largest studies of global public opinion on human rights and democracy over conducted.

The polling, conducted by Savanta as well as local vendors in Ukraine, surveyed participants on questions about democracy and human rights, major issues facing their countries and the world, and international governance.

The report, Open Society Barometer: Can Democracy Deliver?, finds that young people around the world hold the least faith in democracy of any age group.  

While the findings suggest that the concept of democracy remains widely popular, and a vast majority want to live in a democratic state, people cited a number of serious concerns that impact their daily life—from climate change to political violence or simply affording enough food to eat. At this critical turning point, the question becomes: can democracy deliver what people need most?…(More)”.

Initial policy considerations for generative artificial intelligence

Curated on Posted on by Stefaan Verhulst

OECD Report: “Generative artificial intelligence (AI) creates new content in response to prompts, offering transformative potential across multiple sectors such as education, entertainment, healthcare and scientific research. However, these technologies also pose critical societal and policy challenges that policy makers must confront: potential shifts in labour markets, copyright uncertainties, and risk associated with the perpetuation of societal biases and the potential for misuse in the creation of disinformation and manipulated content. Consequences could extend to the spreading of mis- and disinformation, perpetuation of discrimination, distortion of public discourse and markets, and the incitement of violence. Governments recognise the transformative impact of generative AI and are actively working to address these challenges. This paper aims to inform these policy considerations and support decision makers in addressing them…(More)”.

The Rapid Growth of Behavioral Science

Curated on Posted on by Stefaan Verhulst

Article by Steve Wendel: “It’s hard to miss the rapid growth of our field: into new sectors, into new countries, and into new collaborations with other fields. Over the years, I’ve sought to better understand that growth by collecting data about our field and sharing the results. A few weeks ago, I launched the most recent effort – a survey for behavioral science & behavioral design practitioners and one for behavioral researchers around the globe. Here, I’ll share a bit about what we’re seeing so far in the data, and ask for your help to spread it more widely.

First, our field has seen rapid growth since 2008 – which is, naturally, when Thaler and Sunstein’s Nudge first came out. The number of teams and practitioners in the space has grown more or less in tandem, though with a recent slowing in the creation of new teams since 2020. The most productive year was 2019, with 59 new teams starting; the subsequent three years have averaged 28 per year[1].

Behavioral science and design practitioners are also increasingly spread around the world. Just a few years ago, it was difficult to find practitioners outside of BeSci centers in the US, UK, and a few other countries. While we are still heavily concentrated in these areas, there are now active practitioners in 72 countries: from Paraguay to Senegal to Bhutan.

Figure 1: Where practitioners are located. Note – the live and interactive map is available on BehavioralTeams.com.

The majority of practitioners (52%) are in full-time behavioral science or behavioral design roles. The rest are working in other disciplines such as product design and marketing in which they aren’t dedicated to BeSci but have the opportunity to apply it in their work (38%). A minority of individuals have BeSci side jobs (9%).

Among respondents thus far, the most common challenge they are facing is making the case for behavioral science with senior leaders in their organizations (63%) and being able to measure the impact of their inventions (65%). Anecdotally, many practitioners in the field complain that they are asked for their recommendations on what to do, but aren’t given the opportunity to follow up and see if those recommendations were implemented or, when implemented, were actually effective.

The survey asks many more questions about the experiences and backgrounds of practitioners, but we’re still gathering data and will release new results when we have them…(More)”.

Disinformation and Civic Tech Research

Curated on Posted on by Stefaan Verhulst

Code for All Playbook: “”The Disinformation and Civic Tech Playbook is a tool for people who are interested in understanding how civic tech can help confront disinformation. This guide will help you successfully advocate for, and implement disinfo-fighting tools, programs, and campaigns from partners around the world.

In order to effectively fight misinformation at a societal scale, three stages of work must be completed in sequential order:

  1. Monitor or research media environment (traditional, social, and/or messaging apps) for misinformation
  2. Verify and/or debunk
  3. Reach people with the truth and counter-message falsehoods

These stages ascend from least impactful to most impactful activity.

Researching misinformation in the media environment has no effect whatsoever on its own. Verifying and debunking falsehoods have limited utility unless stage three is also achieved: successfully reaching communities with true information in a way that gets through to them, and effectively counter-messaging the misinformation that spreads so easily.

Unfortunately, the distribution of misinformation management projects to date seems to be the exact inverse of these stages. There has been an enormous amount of work to passively monitor and research media environments for misinformation. There is also a large amount of energy and resources dedicated to verifying and debunking misinformation through traditional fact-checking approaches. Whether because it’s the hardest one to solve or just third in the consecutive sequence, relatively few misinformation management projects have made it to the final stage of genuinely getting through to people and experimenting with effective counter-messaging and counter-engagement (see The Sentinel Project interview for further discussion)…(More)”.

Supporting decision making with strategic foresight

Curated on Posted on by Stefaan Verhulst

OECD working paper: “… discusses strategic foresight initiatives and methodologies that support decision making and process design. It highlights case studies, international benchmarks, and best practices, as well as methodological recommendations and options for promoting the adoption and use of strategic foresight in government. The paper has four sections, each centred on a critical action to improve decision making through strategic foresight: (i) framing strategic foresight, (ii) building its fundamental components in governments, (iii) fine-tuning foresight interventions to specific contexts, and (iv) undertaking concrete activities to solve specific policy challenges. Given its exploratory nature, this working paper and its proposals should be seen as contributing to ongoing debates about the use of strategic foresight for decision making in government. The ultimate purpose of this paper is to help governments become more proactive and prospective…(More)”.

Centers of Progress: 40 Cities That Changed the World

Curated on Posted on by Stefaan Verhulst

Book by Chelsea Follett: “Where does progress happen? The story of civilization is the story of the city. It is cities that have created and defined the modern world by acting as the sites of pivotal advances in culture, politics, science, technology, and more. There is no question that certain places, at certain times in history, have contributed disproportionately toward making the world a better place. This book tells the story of forty of those places.

In Centers of Progress: 40 Cities That Changed the World, Chelsea Follett examines a diverse group of cities, ranging from ancient Athens to Song‐​era Hangzhou. But some common themes stand out: most cities reach their creative peak during periods of peace; most centers of progress also thrive during times of social, intellectual, and economic freedom, as well as openness to intercultural exchange and trade; and centers of progress tend to be highly populated. Because, in every city, it is ultimately the people who live there who drive progress forward―if given the freedom to do so.

Identifying common factors―such as relative peace, freedom, and multitudes―among the places that have produced history’s greatest achievements is one way to learn what causes progress. Change is a constant, but progress is not. Understanding what makes a place fertile ground for progress may help to sow the seeds of future innovations.

Moreover, their story is our story. City air provides the wind in the sails of the modern world. Come journey through these pages to some of history’s greatest centers of progress…(More)”.

EU Parliament pushes for more participatory tools for Europeans

Curated on Posted on by Stefaan Verhulst

Article by Silvia Ellena: “A majority of EU lawmakers adopted a report on Thursday (14 September) calling for more participatory tools at EU level. The report, which has no direct legislative impact, passed with 316 votes in favour, 137 against and 47 abstentions.

“We send a clear message to upgrade our democracy, a new EU Agora that involves citizens in European democratic life,” said Alin Mituța (Renew), co-rapporteur on the file, following the adoption of the report.

In the report, the Parliament called for the creation of a European Agora, an annual “structured participation mechanism” composed of citizens, who would deliberate on the EU’s priorities for the year ahead, providing input for the Commission work plan.

Moreover, EU lawmakers called for the creation of a one-stop-shop for all the existing instruments to make sure citizens have easier access to them.

The report also encourages increased use of mini-publics as well as the institutionalisation of other deliberative processes, such as the European Citizens’ Panels, which were set up by the Commission as a follow-up to the EU-wide democratic experiment known as the Conference on the Future of Europe (CoFoE).

These panels, made of randomly selected citizens, were called to deliberate on upcoming legislation earlier this year.

Other participatory tools suggested in the report include EU-wide referendums on key EU policies as well as pan-European online citizens’ consultations to increase citizens’ knowledge of the EU as well as their trust in EU decision-making.

Finally, the Parliament called for an increased focus on the impact of EU policies on youth, suggesting the use of the ‘youth check’, a monitoring tool which has been promoted by the European Youth Forum and included in the CoFoE recommendations.

Other European institutions are already experimenting with the youth check, such as the European Economic and Social Committee (EESC), whose recently appointed president included a youth test among the priorities for his mandate…

According to EU lawmakers, citizens’ participation plays a key role in strengthening democracy and the EU Commission should develop a “comprehensive European strategy to enhance citizenship competences in the EU”…(More)”.

Enhancing the security of communication infrastructure

Curated on Posted on by Stefaan Verhulst

OECD Report: “The digital security of communication networks is crucial to the functioning of our societies. Four trends are shaping networks, raising digital security implications: i) the increasing criticality of communication networks, ii) increased virtualisation of networks and use of cloud services, iii) a shift towards more openness in networks and iv) the role of artificial intelligence in networks. These trends bring benefits and challenges to digital security. While digital security ultimately depends on the decisions made by private actors (e.g. network operators and their suppliers), the report underlines the role governments can play to enhance the digital security of communication networks. It outlines key policy objectives and actions governments can take to incentivise the adoption of best practices and support stakeholders to reach an optimal level of digital security, ranging from light-touch to more interventionist approaches…(More)”.