Article by Chen Yiru: From early-morning school drop-offs to seniors booking rides to the hospital, from suburban commuters seeking a faster link to the metro to families visiting ancestral graves, Shanghai is rolling out a new kind of public bus — one that’s designed by commuters, and launched only when enough riders request it.
Branded “DZ” for dingzhi, or “customized,” the system invites residents to submit proposed routes through a city-run platform. Others with similar travel needs can opt in or vote, and if demand meets the threshold — typically 15 to 20 passengers per trip — the route goes live.
More than 220 DZ routes have already launched across all 16 city districts. Through an online platform opened May 8, users enter start and end points, preferred times, and trip frequency. If approved, routes can begin running in as little as three days…(More)”.
Paper by John Michael Maxel Okoche et al: “Despite significant technology advances especially in artificial intelligence (AI), crowdsourcing platforms still struggle with issues such as data overload and data quality problems, which hinder their full potential. This study addresses a critical gap in the literature how the integration of AI technologies in crowdsourcing could help overcome some these challenges. Using a systematic literature review of 77 journal papers, we identify the key limitations of current crowdsourcing platforms that included issues of quality control, scalability, bias, and privacy. Our research highlights how different forms of AI including from machine learning (ML), deep learning (DL), natural language processing (NLP), automatic speech recognition (ASR), and natural language generation techniques (NLG) can address the challenges most crowdsourcing platforms face. This paper offers knowledge to support the integration of AI first by identifying types of crowdsourcing applications, their challenges and the solutions AI offers for improvement of crowdsourcing…(More)”.
Paper by Zhen-Song Chen and Zheng Ma: “Ensuring fair and accurate hiring outcomes is critical for both job seekers’ economic opportunities and organizational development. This study addresses the challenge of mitigating biases in AI-powered resume screening systems by leveraging crowd intelligence, thereby enhancing problem-solving efficiency and decision-making quality. We propose a novel counterfactual resume-annotation method based on a causal model to capture and correct biases from human resource (HR) representatives, providing robust ground truth data for supervised machine learning. The proposed model integrates multiple language embedding models and diverse HR-labeled data to train a cohort of resume-screening agents. By training 60 such agents with different models and data, we harness their crowd intelligence to optimize for three objectives: accuracy, fairness, and a balance of both. Furthermore, we develop a binary bias-detection model to visualize and analyze gender bias in both human and machine outputs. The results suggest that harnessing crowd intelligence using both accuracy and fairness objectives helps AI systems robustly output accurate and fair results. By contrast, a sole focus on accuracy may lead to severe fairness degradation, while, conversely, a sole focus on fairness leads to a relatively minor loss of accuracy. Our findings underscore the importance of balancing accuracy and fairness in AI-powered resume-screening systems to ensure equitable hiring outcomes and foster inclusive organizational development…(More)”
Excerpt from Lizhi Liu’s new book, “From Click to Boom”: “When disputes occur, Taobao encourages buyers and sellers to negotiate with each other first. If the feuding parties cannot reach an agreement and do not want to go to court, they can use one of Taobao’s two judicial channels: asking a Taobao employee to adjudicate or using an online jury panel to arbitrate. This section discusses the second channel, a unique Chinese institutional innovation.
Alibaba’s Public Jury was established in 2012 to crowdsource justice. It uses a Western-style jury-voting mechanism to solve online disputes and controversial issues. These jurors are termed “public assessors” by Taobao. Interestingly, the name “public assessor” was drawn from the Chinese talent show “Super Girl” (similar to “American Idol”), which, after the authority shut down its mass voting system, transitioned to using a small panel of audience representatives (or “public assessors”) to vote for the show’s winner. The public jury was widely used by the main Taobao site by 2020 and is now frequently used by Xianyu, Taobao’s used-goods market.
Why did Taobao introduce the jury system? Certainly, as Taobao expanded, the volume of online disputes surged, posing challenges for the platform to handle all disputes by itself. However, according to a former platform employee responsible for designing this institution, the primary motivation was not the caseload. Instead, it was propelled by the complexity of online disputes that proved challenging for the platform to resolve alone. Consequently, they opted to involve users in adjudicating these cases to ensure a fairer process rather than solely relying on platform intervention.
To form a jury, Taobao randomly chooses each panel of 13 jurors from 4 million volunteer candidates; each juror may participate in up to 40 cases per day. The candidate needs to be an experienced Taobao user (i.e., those who have registered for more than a year) with a good online reputation (i.e., those who have a sufficiently high credit rating, as discussed below). This requirement is high enough to prevent most dishonest traders from manipulating votes, but low enough to be inclusive and keep the juror pool large. These jurors are unpaid yet motivated to participate. They gain experience points that can translate into different virtual titles or that can be donated to charity by Taobao as real money…(More)”
Book by Marion K. Poetz and Henry Sauermann: “This book explores how millions of people can significantly contribute to scientific research with their effort and experience, even if they are not working at scientific institutions and may not have formal scientific training.
Drawing on a strong foundation of scholarship on crowd involvement, this book helps researchers recognize and understand the benefits and challenges of crowd involvement across key stages of the scientific process. Designed as a practical toolkit, it enables scientists to critically assess the potential of crowd participation, determine when it can be most effective, and implement it to achieve meaningful scientific and societal outcomes.
The book also discusses how recent developments in artificial intelligence (AI) shape the role of crowds in scientific research and can enhance the effectiveness of crowd science projects…(More)”
Paper by Donia Nciri et al: “Traditionally, government and national mapping agencies have been a primary provider of authoritative geospatial information. Today, with the exponential proliferation of Information and Communication Technologies or ICTs (such as GPS, mobile mapping and geo-localized web applications, social media), any user becomes able to produce geospatial information. This participatory production of geographical data gives birth to the concept of Volunteered Geographic Information (VGI). This phenomenon has greatly contributed to the production of huge amounts of heterogeneous data (structured data, textual documents, images, videos, etc.). It has emerged as a potential source of geographic information in many application areas. Despite the various advantages associated with it, this information lacks often quality assurance, since it is provided by diverse user profiles. To address this issue, numerous research studies have been proposed to assess VGI quality in order to help extract relevant content. This work attempts to provide an overall review of VGI quality assessment methods over the last decade. It also investigates varied quality assessment attributes adopted in recent works. Moreover, it presents a classification that forms a basis for future research. Finally, it discusses in detail the relevance and the main limitations of existing approaches and outlines some guidelines for future developments…(More)”.
In February 2016, social entrepreneur Bola Lawal turned to crowdfunding to make educational opportunities accessible to Nigerians. He founded ScholarX as the vehicle to realize this mission through taking advantage of the largely untapped market of unclaimed scholarships, educational grants, and philanthropic donations for African students. The X in ScholarX represents the missing value and recognition that Nigerian youth deserve for their dedication to academic achievement.
“The idea for ScholarX came from the conversation with my friends on our shared experiences,” Lawal recounts, “because I also had difficulty paying for school like millions of Nigerians.” He adds that he was “even suspended from school because” of his inability to pay the tuition fee.
Like Lawal, more than 100,000 Nigerian students overseas rely on scholarships, many of which are backed either by oil and gas companies that aim to recruit students into the industry or by federal government grants for local students. But in recent years, these scholarships have been scaled back or scrapped altogether because of the ongoing economic crisis and recession. The crash of the foreign exchange rate of Nigeria’s currency, the naira, has further threatened the prospects of Nigeria’s overseas students, leaving many unable to pay tuition…(More)”
By: Roshni Singh, Hannah Chafetz, and Stefaan G. Verhulst
The questions that society asks can transform public policy making, mobilize resources, and shape public discourse, yet decision makers around the world frequently focus on developing solutions rather than identifying the questions that need to be addressed to develop those solutions.
This blog provides a range of resources on the potential of questions for society. It includes readings on new approaches to formulating questions, how questions benefit public policy making and democracy, the importance of increasing the capacity for questioning at the individual level, and the role of questions in the age of AI and prompt engineering.
These readings underscore the need for a new science of questions – a new discipline solely focused on integrating participatory approaches for identifying, prioritizing, and addressing questions for society. This emerging discipline not only fosters creativity and critical thinking within societies but also empowers individuals and communities to engage actively in the questioning process, thereby promoting a more inclusive and equitable approach to addressing today’s societal challenges.
A few key takeaways from these readings:
Incorporating participatory approaches in questioning processes: Several of the readings discuss the value of including participatory approaches in questioning as a means to incorporate diverse perspectives, identify where there knowledge gaps, and ensure the questions prioritized reflect current needs. In particular, the readings emphasize the role of open innovation and co-creation principles, workshops, surveys, as ways to make the questioning process more collaborative.
Advancing individuals’ questioning capability: Teaching individuals to ask their own questions fosters agency and is essential for effective democratic participation. The readings recommend cultivating this skill from early education through adulthood to empower individuals to engage actively in decision-making processes.
Improving questioning processes for responsible AI use: In the era of AI and prompt engineering, how questions are framed is key for deriving meaningful responses to AI queries. More focus on participatory question formulation in the context of AI can help foster more inclusive and responsible data governance.
In “Crowdsourcing Research Questions in Science,” the authors examine how involving the general public in formulating research questions can enhance scientific inquiry. They analyze two crowdsourcing projects in the medical sciences and find that crowd-generated questions often restate problems but provide valuable cross-disciplinary insights. Although these questions typically rank lower in novelty and scientific impact compared to professional questions, they match the practical impact of professional research. The authors argue that crowdsourcing can improve research by offering diverse perspectives. They emphasize the importance of using effective selection methods to identify and prioritize the most valuable contributions from the crowd, ensuring that the highest quality questions are highlighted and addressed.
This journal article emphasizes the growing importance of openness and collaboration in scientific research. The authors identify the lack of a unified understanding of these practices due to differences in disciplinary approaches and propose an Open Innovation in Science (OIS) Research Framework (co-developed with 47 scholars) to bridge these knowledge gaps and synthesize information across fields. The authors argue that integrating Open Science and Open Innovation concepts can enhance researchers’ and practitioners’ understanding of how these practices influence the generation and dissemination of scientific insights and innovation. The article highlights the need for interdisciplinary collaboration to address the complexities of societal, technical, and environmental challenges and provides a foundation for future research, policy discussions, and practical guidance in promoting open and collaborative scientific practices.
This figure from Beck et al., Industry and Innovation, 2022, outlines the Open Innovation in Science (OIS) framework, which connects scientific research with societal impacts through an iterative process. It highlights how feedback from scientific and societal outcomes influences research problems, boundary conditions, and antecedents, emphasizing continuous collaboration and openness in the research process.
In “The Surprising Power of Questions,” published in Harvard Business Review, Alison Wood Brooks and Leslie K. John highlight how asking questions drives learning, innovation, and relationship building within organizations. They argue that many executives focus on answers but underestimate how well-crafted questions can enhance communication, build trust, and uncover risks. Drawing from behavioral science, the authors show how the type, tone, and sequence of questions influence the effectiveness of conversations. By refining their questioning skills, individuals can boost emotional intelligence, foster deeper connections, and unlock valuable insights that benefit both themselves and their organizations.
The chart titled “Conversational Goals Matter” from “The Surprising Power of Questions” by Alison Wood Brooks and Leslie K. John (Harvard Business Review, May-June 2018) highlights tactics for handling competitive and cooperative conversations. It outlines strategies like asking direct questions to avoid evasive answers in competitive discussions, and using open-ended questions and building rapport in cooperative conversations. The chart offers practical approaches to improve communication and overcome common conversational challenges.
In “Choosing Policy-Relevant Research Questions,” Paul Kellner explains how social scientists can craft research questions that better inform policy decisions. He highlights the ongoing issue of social sciences not significantly impacting policy, as noted by experts like William Julius Wilson and Christopher Whitty. The article suggests methods for engaging policymakers in the research question formulation process, such as user engagement, co creation, surveys, voting, and consensus-building workshops. Kellner provides examples where policymakers directly participated in the research, resulting in more practical and relevant outcomes. He concludes that improving coordination between researchers and policymakers can enhance the policy impact of social science research.
In this Op-Ed, Andrew P. Minigan emphasizes the critical role of curiosity and question formulation in education. He argues that alongside the “4 Cs” (creativity, critical thinking, communication, and collaboration), there should be a fifth C: curiosity. Asking questions enables students to identify knowledge gaps, think critically and creatively, and engage with peers. Research links curiosity to improved memory, academic achievement, and creativity. Despite these benefits, traditional teaching models often overlook curiosity. Minigan suggests teaching students to formulate questions to boost their curiosity and support educational goals. He concludes that nurturing curiosity is essential for developing innovative thinkers who can explore new, complex questions.
In this blog, Dan Rothstein highlights the importance of fostering “agency,” which is the ability of individuals to think and act independently, as a cornerstone of democracy. Rothstein and his colleague Luz Santana have spent over two decades at The Right Question Institute teaching people how to ask their own questions to enhance their participation in decision-making. They discovered that the inability to ask questions hinders involvement in decisions that impact individuals. Rothstein argues that learning to formulate questions is essential for developing agency and effective democratic participation. This skill should be taught from early education through adulthood. Despite its importance, many students do not learn this in college, so educators must focus on teaching question formulation at all levels. Rothstein concludes that empowering individuals to ask questions is vital for a strong democracy and should be a continuous effort across society.
In the chapter “From a Policy Problem to a Research Question: Getting It Right Together” from the Science for Policy Handbook, Marta Sienkiewicz emphasizes the importance of co-creation between researchers and policymakers to determine relevant research questions. She highlights the need for this approach due to the separation between research and policy cultures, and the differing natures of scientific (tame) and policy (wicked) problems. Sienkiewicz outlines a skills framework and provides examples from the Joint Research Centre (JRC), such as Knowledge Centres, staff exchanges, and collaboration facilitators, to foster interaction and collaboration. Engaging policymakers in the research question development process leads to more practical and relevant outcomes, builds trust, and strengthens relationships. This collaborative approach ensures that research is aligned with policy needs, increases the chances of evidence being used effectively in decision-making, and ultimately enhances the impact of scientific research on policy.
In “Methods for Collaboratively Identifying Research Priorities and Emerging Issues in Science and Policy,” the authors, William J. Sutherland et al., emphasize the importance of bridging the gap between scientific research and policy needs through collaborative approaches. They outline a structured, inclusive methodology that involves researchers, policymakers, and practitioners to jointly identify priority research questions. The approach includes gathering input from diverse stakeholders, iterative voting processes, and structured workshops to refine and prioritize questions, ensuring that the resulting research addresses critical societal and environmental challenges. These methods foster greater collaboration and ensure that scientific research is aligned with the practical needs of policymakers, thereby enhancing the relevance and impact of the research on policy decisions. This approach has been successfully applied in multiple fields, including conservation and agriculture, demonstrating its versatility in addressing both emerging issues and long-term policy priorities.
In this article co-authored with Anil Ananthaswamy, , Stefaan Verhulst emphasizes the crucial role of framing questions correctly, particularly in the era of AI and data. They highlight how ChatGPT’s success underscores the power of well-formulated questions and their impact on deriving meaningful answers. Verhulst and Ananthaswamy argue that society’s focus on answers has overshadowed the importance of questioning, which shapes scientific inquiry, public policy, and data utilization. They call for a new science of questions that integrates diverse fields and promotes critical thinking, data literacy, and inclusive questioning to address biases and improve decision-making. This interdisciplinary effort aims to shift the emphasis from merely seeking answers to understanding the context and purpose behind the questions.
Image of teachers are seen behind a laptop during a workshop on ChatGPT bot in the Swiss canton of Geneva from Fabrice Coffrini
In this chapter published in “Global Digital Data Governance: Polycentric Perspectives”, Stefaan Verhulst explores the crucial role of formulating questions in ensuring responsible data usage. Verhulst argues that, in our data-driven society, responsibly handling data is key to maximizing public good and minimizing risks. He proposes a polycentric approach where the right questions are co-defined to enhance the social impact of data science. Drawing from both conceptual and practical knowledge, including his experience with The 100 Questions Initiative, Verhulst emphasizes that a participatory methodology in question formulation can democratize data use, ensuring data minimization, proportionality, participation, and accountability. By shifting from a supply-driven to a demand-driven approach, Verhulst envisions a new “science of questions” that complements data science, fostering a more inclusive and responsible data governance framework.
As we navigate the complexities of our rapidly changing world, the importance of asking the right questions cannot be overstated. We invite researchers, educators, policymakers, and curious minds alike to delve deeper into new approaches for questioning. By fostering an environment that values and prioritizes well-crafted questions, we can drive innovation, enhance education, improve public policy, and harness the potential of AI and data science. In the coming months, The GovLab, with the support of the Henry Luce Foundation, will be exploring these topics further through a series of roundtable discussions. Are you working on participatory approaches to questioning and are interested in getting involved? Email Stefaan G. Verhulst, Co-Founder and Chief R&D at The GovLab, at [email protected].
Paper by Susanne Beck, Dilek Fraisl, Marion Poetz and Henry Sauermann: “Research on Open Innovation in Science (OIS) investigates how open and collaborative practices influence the scientific and societal impact of research. Since 2019, the OIS Research Conference has brought together scholars and practitioners from diverse backgrounds to discuss OIS research and case examples. In this meeting report, we describe four session formats that have allowed our multi-disciplinary community to have productive discussions around opportunities and challenges related to citizen involvement in research. However, these sessions also highlight the need for a better understanding of the underlying rationales of citizen involvement in an increasingly diverse project landscape. Building on the discussions at the 2023 and prior editions of the conference, we outline a conceptual framework of five crowd paradigms and present an associated tool that can aid in understanding how citizen involvement in particular projects can help advance science. We illustrate this tool using cases presented at the 2023 conference, and discuss how it can facilitate discussions at future conferences as well as guide future research and practice in citizen science…(More)”.
Article by Ramsha Jahangir, Elodie Vialle and Dylan Moses: “It’s been 100 days since the Digital Services Act (DSA) came into effect, and many of us are still wondering how the Trusted Flagger mechanism is taking shape, particularly for civil society organizations (CSOs) that could be potential applicants.
With an emphasis on accountability and transparency, the DSA requires national coordinators to appoint Trusted Flaggers, who are designated entities whose requests to flag illegal content must be prioritized. “Notices submitted by Trusted Flaggers acting within their designated area of expertise . . . are given priority and are processed and decided upon without undue delay,” according to the DSA. Trusted flaggers can include non-governmental organizations, industry associations, private or semi-public bodies, and law enforcement agencies. For instance, a private company that focuses on finding CSAM or terrorist-type content, or tracking groups that traffic in that content, could be eligible for Trusted Flagger status under the DSA. To be appointed, entities need to meet certain criteria, including being independent, accurate, and objective.
Trusted escalation channels are a key mechanism for civil society organizations (CSOs) supporting vulnerable users, such as human rights defenders and journalists targeted by online attacks on social media, particularly in electoral contexts. However, existing channels could be much more efficient. The DSA is a unique opportunity to redesign these mechanisms for reporting illegal or harmful content at scale. They need to be rethought for CSOs that hope to become Trusted Flaggers. Platforms often require, for instance, content to be translated into English and context to be understood by English-speaking audiences (due mainly to the fact that the key decision-makers are based in the US), which creates an added burden for CSOs that are resource-strapped. The lack of transparency in the reporting process can be distressing for the victims for whom those CSOs advocate. The lack of timely response can lead to dramatic consequences for human rights defenders and information integrity. Several CSOs we spoke with were not even aware of these escalation channels – and platforms are not incentivized to promote mechanisms given the inability to vet, prioritize and resolve all potential issues sent to them….(More)”.
