Paper by Louis Rosenberg et al: “Conversational Swarm Intelligence (CSI) is an AI-powered communication and collaboration technology that allows large, networked groups (of potentially unlimited size) to hold thoughtful conversational deliberations in real-time. Inspired by the efficient decision-making dynamics of fish schools, CSI divides a human population into a set of small subgroups connected by AI agents. This enables the full group to hold a unified conversation. In this study, groups of 25 participants were tasked with selecting a roster of players in a real Fantasy Baseball contest. A total of 10 trials were run using CSI. In half the trials, each subgroup was augmented with a fact-providing AI agent referred to herein as an Infobot. The Infobot was loaded with a wide range of MLB statistics. The human participants could query the Infobot the same way they would query other persons in their subgroup. Results show that when using CSI, the 25-person groups outperformed 72% of individually surveyed participants and showed significant intelligence amplification versus the mean score (p=0.016). The CSI-enabled groups also significantly outperformed the most popular picks across the collected surveys for each daily contest (p<0.001). The CSI sessions that used Infobots scored slightly higher than those that did not, but it was not statistically significant in this study. That said, 85% of participants agreed with the statement ‘Our decisions were stronger because of information provided by the Infobot’ and only 4% disagreed. In addition, deliberations that used Infobots showed significantly less variance (p=0.039) in conversational content across members. This suggests that Infobots promoted more balanced discussions in which fewer members dominated the dialog. This may be because the infobot enabled participants to confidently express opinions with the support of factual data…(More)”.
When combinations of humans and AI are useful: A systematic review and meta-analysis
Paper by Michelle Vaccaro, Abdullah Almaatouq & Thomas Malone: “Inspired by the increasing use of artificial intelligence (AI) to augment humans, researchers have studied human–AI systems involving different tasks, systems and populations. Despite such a large body of work, we lack a broad conceptual understanding of when combinations of humans and AI are better than either alone. Here we addressed this question by conducting a preregistered systematic review and meta-analysis of 106 experimental studies reporting 370 effect sizes. We searched an interdisciplinary set of databases (the Association for Computing Machinery Digital Library, the Web of Science and the Association for Information Systems eLibrary) for studies published between 1 January 2020 and 30 June 2023. Each study was required to include an original human-participants experiment that evaluated the performance of humans alone, AI alone and human–AI combinations. First, we found that, on average, human–AI combinations performed significantly worse than the best of humans or AI alone (Hedges’ g = −0.23; 95% confidence interval, −0.39 to −0.07). Second, we found performance losses in tasks that involved making decisions and significantly greater gains in tasks that involved creating content. Finally, when humans outperformed AI alone, we found performance gains in the combination, but when AI outperformed humans alone, we found losses. Limitations of the evidence assessed here include possible publication bias and variations in the study designs analysed. Overall, these findings highlight the heterogeneity of the effects of human–AI collaboration and point to promising avenues for improving human–AI systems…(More)”.
Orphan Articles: The Dark Matter of Wikipedia
Paper by Akhil Arora, Robert West, Martin Gerlach: “With 60M articles in more than 300 language versions, Wikipedia is the largest platform for open and freely accessible knowledge. While the available content has been growing continuously at a rate of around 200K new articles each month, very little attention has been paid to the accessibility of the content. One crucial aspect of accessibility is the integration of hyperlinks into the network so the articles are visible to readers navigating Wikipedia. In order to understand this phenomenon, we conduct the first systematic study of orphan articles, which are articles without any incoming links from other Wikipedia articles, across 319 different language versions of Wikipedia. We find that a surprisingly large extent of content, roughly 15\% (8.8M) of all articles, is de facto invisible to readers navigating Wikipedia, and thus, rightfully term orphan articles as the dark matter of Wikipedia. We also provide causal evidence through a quasi-experiment that adding new incoming links to orphans (de-orphanization) leads to a statistically significant increase of their visibility in terms of the number of pageviews. We further highlight the challenges faced by editors for de-orphanizing articles, demonstrate the need to support them in addressing this issue, and provide potential solutions for developing automated tools based on cross-lingual approaches. Overall, our work not only unravels a key limitation in the link structure of Wikipedia and quantitatively assesses its impact, but also provides a new perspective on the challenges of maintenance associated with content creation at scale in Wikipedia…(More)”.
AI-enhanced collective intelligence
Paper by Hao Cui and Taha Yasseri: “Current societal challenges exceed the capacity of humans operating either alone or collectively. As AI evolves, its role within human collectives will vary from an assistive tool to a participatory member. Humans and AI possess complementary capabilities that, together, can surpass the collective intelligence of either humans or AI in isolation. However, the interactions in humanAI systems are inherently complex, involving intricate processes and interdependencies. This review incorporates perspectives from complex network science to conceptualize a multilayer representation of human-AI collective intelligence, comprising cognition, physical, and information layers. Within this multilayer network, humans and AI agents exhibit varying characteristics; humans differ in diversity from surface-level to deep-level attributes, while AI agents range in degrees of functionality and anthropomorphism. We explore how agents’ diversity and interactions influence the system’s collective intelligence and analyze real-world instances of AI-enhanced collective intelligence. We conclude by considering potential challenges and future developments in this field….(More)” See also: Where and When AI and CI Meet: Exploring the Intersection of Artificial and Collective Intelligence
Selected Readings: Exploring the Power of Questions For Society
By: Roshni Singh, Hannah Chafetz, and Stefaan G. Verhulst
The questions that society asks can transform public policy making, mobilize resources, and shape public discourse, yet decision makers around the world frequently focus on developing solutions rather than identifying the questions that need to be addressed to develop those solutions.
This blog provides a range of resources on the potential of questions for society. It includes readings on new approaches to formulating questions, how questions benefit public policy making and democracy, the importance of increasing the capacity for questioning at the individual level, and the role of questions in the age of AI and prompt engineering.
These readings underscore the need for a new science of questions – a new discipline solely focused on integrating participatory approaches for identifying, prioritizing, and addressing questions for society. This emerging discipline not only fosters creativity and critical thinking within societies but also empowers individuals and communities to engage actively in the questioning process, thereby promoting a more inclusive and equitable approach to addressing today’s societal challenges.
A few key takeaways from these readings:
- Incorporating participatory approaches in questioning processes: Several of the readings discuss the value of including participatory approaches in questioning as a means to incorporate diverse perspectives, identify where there knowledge gaps, and ensure the questions prioritized reflect current needs. In particular, the readings emphasize the role of open innovation and co-creation principles, workshops, surveys, as ways to make the questioning process more collaborative.
- Advancing individuals’ questioning capability: Teaching individuals to ask their own questions fosters agency and is essential for effective democratic participation. The readings recommend cultivating this skill from early education through adulthood to empower individuals to engage actively in decision-making processes.
- Improving questioning processes for responsible AI use: In the era of AI and prompt engineering, how questions are framed is key for deriving meaningful responses to AI queries. More focus on participatory question formulation in the context of AI can help foster more inclusive and responsible data governance.
***
In “Crowdsourcing Research Questions in Science,” the authors examine how involving the general public in formulating research questions can enhance scientific inquiry. They analyze two crowdsourcing projects in the medical sciences and find that crowd-generated questions often restate problems but provide valuable cross-disciplinary insights. Although these questions typically rank lower in novelty and scientific impact compared to professional questions, they match the practical impact of professional research. The authors argue that crowdsourcing can improve research by offering diverse perspectives. They emphasize the importance of using effective selection methods to identify and prioritize the most valuable contributions from the crowd, ensuring that the highest quality questions are highlighted and addressed.
This journal article emphasizes the growing importance of openness and collaboration in scientific research. The authors identify the lack of a unified understanding of these practices due to differences in disciplinary approaches and propose an Open Innovation in Science (OIS) Research Framework (co-developed with 47 scholars) to bridge these knowledge gaps and synthesize information across fields. The authors argue that integrating Open Science and Open Innovation concepts can enhance researchers’ and practitioners’ understanding of how these practices influence the generation and dissemination of scientific insights and innovation. The article highlights the need for interdisciplinary collaboration to address the complexities of societal, technical, and environmental challenges and provides a foundation for future research, policy discussions, and practical guidance in promoting open and collaborative scientific practices.
In “The Surprising Power of Questions,” published in Harvard Business Review, Alison Wood Brooks and Leslie K. John highlight how asking questions drives learning, innovation, and relationship building within organizations. They argue that many executives focus on answers but underestimate how well-crafted questions can enhance communication, build trust, and uncover risks. Drawing from behavioral science, the authors show how the type, tone, and sequence of questions influence the effectiveness of conversations. By refining their questioning skills, individuals can boost emotional intelligence, foster deeper connections, and unlock valuable insights that benefit both themselves and their organizations.
Kellner, Paul. “Choosing Policy-Relevant Research Questions.” Good Questions Review, May 21, 2024.
In “Choosing Policy-Relevant Research Questions,” Paul Kellner explains how social scientists can craft research questions that better inform policy decisions. He highlights the ongoing issue of social sciences not significantly impacting policy, as noted by experts like William Julius Wilson and Christopher Whitty. The article suggests methods for engaging policymakers in the research question formulation process, such as user engagement, co creation, surveys, voting, and consensus-building workshops. Kellner provides examples where policymakers directly participated in the research, resulting in more practical and relevant outcomes. He concludes that improving coordination between researchers and policymakers can enhance the policy impact of social science research.
In this Op-Ed, Andrew P. Minigan emphasizes the critical role of curiosity and question formulation in education. He argues that alongside the “4 Cs” (creativity, critical thinking, communication, and collaboration), there should be a fifth C: curiosity. Asking questions enables students to identify knowledge gaps, think critically and creatively, and engage with peers. Research links curiosity to improved memory, academic achievement, and creativity. Despite these benefits, traditional teaching models often overlook curiosity. Minigan suggests teaching students to formulate questions to boost their curiosity and support educational goals. He concludes that nurturing curiosity is essential for developing innovative thinkers who can explore new, complex questions.
Rothstein, Dan. “Questions, Agency and Democracy.” Medium (blog), February 25, 2017.
In this blog, Dan Rothstein highlights the importance of fostering “agency,” which is the ability of individuals to think and act independently, as a cornerstone of democracy. Rothstein and his colleague Luz Santana have spent over two decades at The Right Question Institute teaching people how to ask their own questions to enhance their participation in decision-making. They discovered that the inability to ask questions hinders involvement in decisions that impact individuals. Rothstein argues that learning to formulate questions is essential for developing agency and effective democratic participation. This skill should be taught from early education through adulthood. Despite its importance, many students do not learn this in college, so educators must focus on teaching question formulation at all levels. Rothstein concludes that empowering individuals to ask questions is vital for a strong democracy and should be a continuous effort across society.
In the chapter “From a Policy Problem to a Research Question: Getting It Right Together” from the Science for Policy Handbook, Marta Sienkiewicz emphasizes the importance of co-creation between researchers and policymakers to determine relevant research questions. She highlights the need for this approach due to the separation between research and policy cultures, and the differing natures of scientific (tame) and policy (wicked) problems. Sienkiewicz outlines a skills framework and provides examples from the Joint Research Centre (JRC), such as Knowledge Centres, staff exchanges, and collaboration facilitators, to foster interaction and collaboration. Engaging policymakers in the research question development process leads to more practical and relevant outcomes, builds trust, and strengthens relationships. This collaborative approach ensures that research is aligned with policy needs, increases the chances of evidence being used effectively in decision-making, and ultimately enhances the impact of scientific research on policy.
In “Methods for Collaboratively Identifying Research Priorities and Emerging Issues in Science and Policy,” the authors, William J. Sutherland et al., emphasize the importance of bridging the gap between scientific research and policy needs through collaborative approaches. They outline a structured, inclusive methodology that involves researchers, policymakers, and practitioners to jointly identify priority research questions. The approach includes gathering input from diverse stakeholders, iterative voting processes, and structured workshops to refine and prioritize questions, ensuring that the resulting research addresses critical societal and environmental challenges. These methods foster greater collaboration and ensure that scientific research is aligned with the practical needs of policymakers, thereby enhancing the relevance and impact of the research on policy decisions. This approach has been successfully applied in multiple fields, including conservation and agriculture, demonstrating its versatility in addressing both emerging issues and long-term policy priorities.
In this article co-authored with Anil Ananthaswamy, , Stefaan Verhulst emphasizes the crucial role of framing questions correctly, particularly in the era of AI and data. They highlight how ChatGPT’s success underscores the power of well-formulated questions and their impact on deriving meaningful answers. Verhulst and Ananthaswamy argue that society’s focus on answers has overshadowed the importance of questioning, which shapes scientific inquiry, public policy, and data utilization. They call for a new science of questions that integrates diverse fields and promotes critical thinking, data literacy, and inclusive questioning to address biases and improve decision-making. This interdisciplinary effort aims to shift the emphasis from merely seeking answers to understanding the context and purpose behind the questions.
In this chapter published in “Global Digital Data Governance: Polycentric Perspectives”, Stefaan Verhulst explores the crucial role of formulating questions in ensuring responsible data usage. Verhulst argues that, in our data-driven society, responsibly handling data is key to maximizing public good and minimizing risks. He proposes a polycentric approach where the right questions are co-defined to enhance the social impact of data science. Drawing from both conceptual and practical knowledge, including his experience with The 100 Questions Initiative, Verhulst emphasizes that a participatory methodology in question formulation can democratize data use, ensuring data minimization, proportionality, participation, and accountability. By shifting from a supply-driven to a demand-driven approach, Verhulst envisions a new “science of questions” that complements data science, fostering a more inclusive and responsible data governance framework.
Table 1 from Verhulst, Stefaan G. “Questions as a Device for Data Responsibility: Toward a New Science of Questions to Steer and Complement the Use of Data Science for the Public Good in a Polycentric Way,” outlines how questions serve as tools for data responsibility across three principles: minimization and proportionality, participation, and accountability. Questions help determine data collection purposes, develop retention policies, foster inclusive debates, secure social licenses for data re-use, identify stakeholders, create feedback loops, and enhance accountability by anticipating risks.
***
As we navigate the complexities of our rapidly changing world, the importance of asking the right questions cannot be overstated. We invite researchers, educators, policymakers, and curious minds alike to delve deeper into new approaches for questioning. By fostering an environment that values and prioritizes well-crafted questions, we can drive innovation, enhance education, improve public policy, and harness the potential of AI and data science. In the coming months, The GovLab, with the support of the Henry Luce Foundation, will be exploring these topics further through a series of roundtable discussions. Are you working on participatory approaches to questioning and are interested in getting involved? Email Stefaan G. Verhulst, Co-Founder and Chief R&D at The GovLab, at sverhulst@thegovlab.org.
DAOs of Collective Intelligence? Unraveling the Complexity of Blockchain Governance in Decentralized Autonomous Organizations
Paper by Mark C. Ballandies, Dino Carpentras, and Evangelos Pournaras: “Decentralized autonomous organizations (DAOs) have transformed organizational structures by shifting from traditional hierarchical control to decentralized approaches, leveraging blockchain and cryptoeconomics. Despite managing significant funds and building global networks, DAOs face challenges like declining participation, increasing centralization, and inabilities to adapt to changing environments, which stifle innovation. This paper explores DAOs as complex systems and applies complexity science to explain their inefficiencies. In particular, we discuss DAO challenges, their complex nature, and introduce the self-organization mechanisms of collective intelligence, digital democracy, and adaptation. By applying these mechansims to improve DAO design and construction, a practical design framework for DAOs is created. This contribution lays a foundation for future research at the intersection of complexity science and DAOs…(More)”.
How to build a Collective Mind that speaks for humanity in real-time
Blog by Louis Rosenberg: “This begs the question — could large human groups deliberate in real-time with the efficiency of fish schools and quickly reach optimized decisions?
For years this goal seemed impossible. That’s because conversational deliberations have been shown to be most productive in small groups of 4 to 7 people and quickly degrade as groups grow larger. This is because the “airtime per person” gets progressively squeezed and the wait-time to respond to others steadily increases. By 12 to 15 people, the conversational dynamics change from thoughtful debate to a series of monologues that become increasingly disjointed. By 20 people, the dialog ceases to be a conversation at all. This problem seemed impenetrable until recent advances in Generative AI opened up new solutions.
The resulting technology is called Conversational Swarm Intelligence and it promises to allow groups of almost any size (200, 2000, or even 2 million people) to discuss complex problems in real-time and quickly converge on solutions with significantly amplified intelligence. The first step is to divide the population into small subgroups, each sized for thoughtful dialog. For example, a 1000-person group could be divided into 200 subgroups of 5, each routed into their own chat room or video conferencing session. Of course, this does not create a single unified conversation — it creates 200 parallel conversations…(More)”.
Using Artificial Intelligence to Accelerate Collective Intelligence
Paper by Róbert Bjarnason, Dane Gambrell and Joshua Lanthier-Welch: “In an era characterized by rapid societal changes and complex challenges, institutions’ traditional methods of problem-solving in the public sector are increasingly proving inadequate. In this study, we present an innovative and effective model for how institutions can use artificial intelligence to enable groups of people to generate effective solutions to urgent problems more efficiently. We describe a proven collective intelligence method, called Smarter Crowdsourcing, which is designed to channel the collective intelligence of those with expertise about a problem into actionable solutions through crowdsourcing. Then we introduce Policy Synth, an innovative toolkit which leverages AI to make the Smarter Crowdsourcing problem-solving approach both more scalable, more effective and more efficient. Policy Synth is crafted using a human-centric approach, recognizing that AI is a tool to enhance human intelligence and creativity, not replace it. Based on a real-world case study comparing the results of expert crowdsourcing alone with expert sourcing supported by Policy Synth AI agents, we conclude that Smarter Crowdsourcing with Policy Synth presents an effective model for integrating the collective wisdom of human experts and the computational power of AI to enhance and scale up public problem-solving processes.
The potential for artificial intelligence to enhance the performance of groups of people has been a topic of great interest among scholars of collective intelligence. Though many AI toolkits exist, they too often are not fitted to the needs of institutions and policymakers. While many existing approaches view AI as a tool to make crowdsourcing and deliberative processes better and more efficient, Policy Synth goes a step further, recognizing that AI can also be used to synthesize the findings from engagements together with research to develop evidence-based solutions and policies. This study contributes significantly to the fields of collective intelligence, public problem-solving, and AI. The study offers practical tools and insights for institutions looking to engage communities effectively in addressing urgent societal challenges…(More)”
Collective Intelligence in Open Policymaking
Book by Rafał Olszowski: “This book examines the nexus of collective intelligence (CI), a feature of online projects in which various types of communities solve problems intelligently, and open policymaking, as a trend of large groups of people shaping public policies.
While doing so, it presents the current state of theoretical knowledge for these concepts, many practical examples of successful and unsuccessful projects, as well as additional research and laboratory experiments. The book develops an analytical framework based on qualitative research, which is applied to the analysis of different projects in selected case studies, such as Decide Madrid; Better Reykjavik; Loomio; Deliberatorium; Civic Budget of the City of Kraków.
The book is structured into four chapters, addressing essential questions in the field: (1) Opening Policymaking; (2) Beyond the Individual: Understanding the Evolution of Collective Intelligence; (3) A Review of the Projects Using Collective Intelligence in Policymaking; (4) Online Public Debate. How Can We Make it More Intelligent?…(More)”.
Untapped
About: “Twenty-first century collective intelligence- combining people’s knowledge and skills, new forms of data and increasingly, technology – has the untapped potential to transform the way we understand and act on climate change.
Collective intelligence for climate action in the Global South takes many forms: from crowdsourcing of indigenous knowledge to preserve biodiversity to participatory monitoring of extreme heat and farmer experiments adapting crops to weather variability.
This research analyzes 100+ climate case studies across 45 countries that tap into people’s participation and use new forms of data. This research illustrates the potential that exists in communities everywhere to contribute to climate adaptation and mitigation efforts. It also aims to shine a light on practical ways in which these initiatives could be designed and further developed so this potential can be fully unleashed…(More)”.