Inquiry as Infrastructure: Defining Good Questions in the Age of Data and AI

Curated on Posted on by Stefaan Verhulst

Paper by Stefaan Verhulst: “The most consequential failures in data-driven policymaking and AI deployment often stem not from poor models or inadequate datasets but from poorly framed questions. This paper centers question literacy as a critical yet underdeveloped competency in the data and policy landscape. Arguing for a “new science of questions,” it explores what constitutes a good question-one that is not only technically feasible but also ethically grounded, socially legitimate, and aligned with real-world needs. Drawing on insights from The GovLab’s 100 Questions Initiative, the paper develops a taxonomy of question types-descriptive, diagnostic, predictive, and prescriptive-and identifies five essential criteria for question quality: questions must be general yet concrete, co-designed with affected communities and domain experts, purpose-driven and ethically sound, grounded in data and technical realities, and capable of evolving through iterative refinement. The paper also outlines common pathologies of bad questions, such as vague formulation, biased framing, and solution-first thinking. Rather than treating questions as incidental to analysis, it argues for institutionalizing deliberate question design through tools like Q-Labs, question maturity models, and new professional roles for data stewards. Ultimately, the paper contends that the questions are infrastructures of meaning. What we ask shapes not only what data we collect or what models we build but also what values we uphold and what futures we make possible…(More)”.

The Overlooked Importance of Data Reuse in AI Infrastructure

Curated on Posted on by Stefaan Verhulst

Essay by Oxford Insights and The Data Tank: “Employing data stewards and embedding responsible data reuse principles in the programme or ecosystem and within participating organisations is one of the pathways forward. Data stewards are proactive agents responsible for catalysing collaboration, tackling these challenges and embedding data reuse practices in their organisations. 

The role of Chief Data Officer for government agencies has become more common in recent years and we suggest the same needs to happen with the role of the Chief Data Steward. Chief Data Officers are mostly focused on internal data management and have a technical focus. With the changes in the data governance landscape, this profession needs to be reimagined and iterated. Embedded in both the demand and the supply sides of data, data stewards are proactive agents empowered to create public value by re-using data and data expertise. They are tasked to identify opportunities for productive cross-sectoral collaboration, and proactively request or enable functional access to data, insights, and expertise. 

One exception comes from New Zealand. The UN has released a report on the role of data stewards and National Statistical Offices (NSOs) in the new data ecosystem. This report provides many use-cases that can be adopted by governments seeking to establish such a role. In New Zealand, there is an appointed Government Chief Data Steward, who is in charge of setting the strategic direction for government’s data management, and focuses on data reuse altogether. 

Data stewards can play an important role in organisations leading data reuse programmes. Data stewards would be responsible for responding to the challenges with participation introduced above. 

A Data Steward’s role includes attracting participation for data reuse programmes by:

  • Demonstrating and communicating the value proposition of data reuse and collaborations, by engaging in partnerships and steering data reuse and sharing among data commons, cooperatives, or collaborative infrastructures. 
  • Developing responsible data lifecycle governance, and communicating insights to raise awareness and build trust among stakeholders; 

A Data Steward’s role includes maintaining and scaling participation for data reuse programmes by:

  • Maintaining trust by engaging with wider stakeholders and establishing clear engagement methodologies. For example, by embedding a social license, data stewards assure the digital self determination principle is embedded in data reuse processes. 
  • Fostering sustainable partnerships and collaborations around data, via developing business cases for data sharing and reuse, and measuring impact to build the societal case for data collaboration; and
  • Innovating in the sector by turning data to decision intelligence to ensure that insights derived from data are more effectively integrated into decision-making processes…(More)”.

Guiding the provision of quality policy advice: the 5D model

Curated on Posted on by Stefaan Verhulst

Paper by Christopher Walker and Sally Washington: “… presents a process model to guide the production of quality policy advice. The work draws on engagement with both public sector practitioners and academics to design a process model for the development of policy advice that works in practice (can be used by policy professionals in their day-to-day work) and aligns with theory (can be taught as part of explaining the dynamics of a wider policy advisory system). The 5D Model defines five key domains of inquiry: understanding Demand, being open to Discovery, undertaking Design, identifying critical Decision points, and shaping advice to enable Delivery. Our goal is a ‘repeatable, scalable’ model for supporting policy practitioners to provide quality advice to decision makers. The model was developed and tested through an extensive process of engagement with senior policy practitioners who noted the heuristic gave structure to practices that determine how policy advice is organized and formulated. Academic colleagues confirmed the utility of the model for explaining and teaching how policy is designed and delivered within the context of a wider policy advisory system (PAS). A unique aspect of this work was the collaboration and shared interest amongst academics and practitioners to define a model that is ‘useful for teaching’ and ‘useful for doing’…(More)”.

From Answer-Giving to Question-Asking: Inverting the Socratic Method in the Age of AI

Curated on Posted on by Stefaan Verhulst

Blog by Anthea Roberts: “…If questioning is indeed becoming a premier cognitive skill in the AI age, how should education and professional development evolve? Here are some possibilities:

  1. Assessment Through Iterative Questioning: Rather than evaluating students solely on their answers, we might assess their ability to engage in sustained, productive questioning—their skill at probing, following up, identifying inconsistencies, and refining inquiries over multiple rounds. Can they navigate a complex problem through a series of well-crafted questions? Can they identify when an AI response contains subtle errors or omissions that require further exploration?
  2. Prompt Literacy as Core Curriculum: Just as reading and writing are foundational literacies, the ability to effectively prompt and question AI systems may become a basic skill taught from early education onward. This would include teaching students how to refine queries, test assumptions, and evaluate AI responses critically—recognizing that AI systems still hallucinate, contain biases from their training data, and have uneven performance across different domains.
  3. Socratic AI Interfaces: Future AI interfaces might be designed explicitly to encourage Socratic dialogue rather than one-sided Q&A. Instead of simply answering queries, these systems might respond with clarifying questions of their own: “It sounds like you’re asking about X—can you tell me more about your specific interest in this area?” This would model the kind of iterative exchange that characterizes productive human-human dialogue…(More)”.

How to Survive the A.I. Revolution

Curated on Posted on by Stefaan Verhulst

Essay by John Cassidy: “It isn’t clear where the term “Luddite” originated. Some accounts trace it to Ned Ludd, a textile worker who reportedly smashed a knitting frame in 1779. Others suggest that it may derive from folk memories of King Ludeca, a ninth-century Anglo-Saxon monarch who died in battle. Whatever the source, many machine breakers identified “General Ludd” as their leader. A couple of weeks after the Rawfolds attack, William Horsfall, another mill owner, was shot dead. A letter sent after Horsfall’s assassination—which hailed “the avenging of the death of the two brave youths who fell at the siege of Rawfolds”—began “By Order of General Ludd.”

The British government, at war with Napoleon, regarded the Luddites as Jacobin insurrectionists and responded with brutal suppression. But this reaction stemmed from a fundamental misinterpretation. Far from being revolutionary, Luddism was a defensive response to the industrial capitalism that was threatening skilled workers’ livelihoods. The Luddites weren’t mindless opponents of technology but had a clear logic to their actions—an essentially conservative one. Since they had no political representation—until 1867, the British voting franchise excluded the vast majority—they concluded that violent protest was their only option. “The burning of Factorys or setting fire to the property of People we know is not right, but Starvation forces Nature to do that which he would not,” one Yorkshire cropper wrote. “We have tried every effort to live by Pawning our Cloaths and Chattles, so we are now on the brink for the last struggle.”

As alarm about artificial intelligence has gone global, so has a fascination with the Luddites. The British podcast “The Ned Ludd Radio Hour” describes itself as “your weekly dose of tech skepticism, cynicism, and absurdism.” Kindred themes are explored in the podcast “This Machine Kills,” co-hosted by the social theorist Jathan Sadowski, whose new book, “The Mechanic and the Luddite,” argues that the fetishization of A.I. and other digital technologies obscures their role in disciplining labor and reinforcing a profit-driven system. “Luddites want technology—the future—to work for all of us,” he told the Guardian.The technology journalist Brian Merchant makes a similar case in “Blood in the Machine: The Origins of the Rebellion Against Big Tech” (2023). Blending a vivid account of the original Luddites with an indictment of contemporary tech giants like Amazon and Uber, Merchant portrays the current wave of automation as part of a centuries-long struggle over labor and power. “Working people are staring down entrepreneurs, tech monopolies, and venture capital firms that are hunting for new forms of labor-saving tech—be it AI, robotics, or software automation—to replace them,” Merchant writes. “They are again faced with losing their jobs to the machine.”..(More)”.

The Future is Coded: How AI is Rewriting the Rules of Decision Theaters

Curated on Posted on by Stefaan Verhulst

Essay by Mark Esposito and David De Cremer: “…These advances are not happening in isolation on engineers’ laptops; they are increasingly playing out in “decision theaters” – specialized environments (physical or virtual) designed for interactive, collaborative problem-solving. A decision theater is typically a space equipped with high-resolution displays, simulation engines, and data visualization tools where stakeholders can convene to explore complex scenarios. Originally pioneered at institutions like Arizona State University, the concept of a decision theater has gained traction as a way to bring together diverse expertise – economists, scientists, community leaders, government officials, and now AI systems – under one roof. By visualizing possible futures (say, the spread of a wildfire or the regional impact of an economic policy) in an engaging, shared format, these theaters make foresight a participatory exercise rather than an academic one. In the age of generative AI, decision theaters are evolving into hubs for human-AI collaboration. Picture a scenario where city officials are debating a climate adaptation policy. Inside a decision theater, an AI model might project several climate futures for the city (varying rainfall, extreme heat incidents, flood patterns) on large screens. Stakeholders can literally see the potential impacts on maps and graphs. They can then ask the AI to adjust assumptions – “What if we add more green infrastructure in this district?” – and within seconds, watch a new projection unfold. This real-time interaction allows for an iterative dialogue between human ideas and AI-generated outcomes. Participants can inject local knowledge or voice community values, and the AI will incorporate that input to revise the scenario. The true power of generative AI in a decision theater lies in this collaboration.

Such interactive environments enhance learning and consensus-building. When stakeholders jointly witness how certain choices lead to undesirable futures (for instance, a policy leading to water shortages in a simulation), it can galvanize agreement on preventative action. Moreover, the theater setup encourages asking “What if?” in a safe sandbox, including ethically fraught questions. Because the visualizations make outcomes concrete, they naturally prompt ethical deliberation: If one scenario shows economic growth but high social inequality, is that future acceptable? If not, how can we tweak inputs to produce a more equitable outcome? In this way, decision theaters embed ethical and social considerations into high-tech planning, ensuring that the focus isn’t just on what is likely or profitable but on what is desirable for communities. This participatory approach helps balance technological possibilities with human values and cultural sensitivities. It’s one thing for an AI to suggest an optimal solution on paper; it’s another to have community representatives in the room, engaging with that suggestion and shaping it to fit local norms and needs.

Equally important, decision theaters democratize foresight. They open up complex decision-making processes to diverse stakeholders, not just technical experts. City planners, elected officials, citizens’ groups, and subject matter specialists can all contribute in real time, aided by AI. This inclusive model guards against the risk of AI becoming an opaque oracle controlled by a few. Instead, the AI’s insights are put on display for all to scrutinize and question. By doing so, the process builds trust in the tools and the decisions that come out of them. When people see that an AI’s recommendation emerged from transparent, interactive exploration – rather than a mysterious black box – they may be more likely to trust and accept the outcome. As one policy observer noted, it’s essential to bring ideas from across sectors and disciplines into these AI-assisted discussions so that solutions “work for people, not just companies.” If designed well, decision theaters operationalize that principle…(More)”.

Mind the (Language) Gap: Mapping the Challenges of LLM Development in Low-Resource Language Contexts

Curated on Posted on by Stefaan Verhulst

White Paper by the Stanford Institute for Human-Centered AI (HAI), the Asia Foundation and the University of Pretoria: “…maps the LLM development landscape for low-resource languages, highlighting challenges, trade-offs, and strategies to increase investment; prioritize cross-disciplinary, community-driven development; and ensure fair data ownership…

  • Large language model (LLM) development suffers from a digital divide: Most major LLMs underperform for non-English—and especially low-resource—languages; are not attuned to relevant cultural contexts; and are not accessible in parts of the Global South.
  • Low-resource languages (such as Swahili or Burmese) face two crucial limitations: a scarcity of labeled and unlabeled language data and poor quality data that is not sufficiently representative of the languages and their sociocultural contexts.
  • To bridge these gaps, researchers and developers are exploring different technical approaches to developing LLMs that better perform for and represent low-resource languages but come with different trade-offs:
    • Massively multilingual models, developed primarily by large U.S.-based firms, aim to improve performance for more languages by including a wider range of (100-plus) languages in their training datasets.
    • Regional multilingual models, developed by academics, governments, and nonprofits in the Global South, use smaller training datasets made up of 10-20 low-resource languages to better cater to and represent a smaller group of languages and cultures.
    • Monolingual or monocultural models, developed by a variety of public and private actors, are trained on or fine-tuned for a single low-resource language and thus tailored to perform well for that language…(More)”

Deliberative Approaches to Inclusive Governance

Curated on Posted on by Stefaan Verhulst

Series edited by Taylor Owen and Sequoia Kim: “Democracy has undergone profound changes over the past decade, shaped by rapid technological, social, and political transformations. Across the globe, citizens are demanding more meaningful and sustained engagement in governance—especially around emerging technologies like artificial intelligence (AI), which increasingly shape the contours of public life.

From world-leading experts in deliberative democracy, civic technology, and AI governance we introduce a seven-part essay series exploring how deliberative democratic processes like citizen’s assemblies and civic tech can strengthen AI governance…(More)”.

Open with care: transparency and data sharing in civically engaged research

Curated on Posted on by Stefaan Verhulst

Paper by Ankushi Mitra: “Research transparency and data access are considered increasingly important for advancing research credibility, cumulative learning, and discovery. However, debates persist about how to define and achieve these goals across diverse forms of inquiry. This article intervenes in these debates, arguing that the participants and communities with whom scholars work are active stakeholders in science, and thus have a range of rights, interests, and researcher obligations to them in the practice of transparency and openness. Drawing on civically engaged research and related approaches that advocate for subjects of inquiry to more actively shape its process and share in its benefits, I outline a broader vision of research openness not only as a matter of peer scrutiny among scholars or a top-down exercise in compliance, but rather as a space for engaging and maximizing opportunities for all stakeholders in research. Accordingly, this article provides an ethical and practical framework for broadening transparency, accessibility, and data-sharing and benefit-sharing in research. It promotes movement beyond open science to a more inclusive and socially responsive science anchored in a larger ethical commitment: that the pursuit of knowledge be accountable and its benefits made accessible to the citizens and communities who make it possible…(More)”.

Artificial Intelligence and Big Data

Curated on Posted on by Stefaan Verhulst

Book edited by Frans L. Leeuw and Michael Bamberger: “…explores how Artificial Intelligence (AI) and Big Data contribute to the evaluation of the rule of law (covering legal arrangements, empirical legal research, law and technology, and international law), and social and economic development programs in both industrialized and developing countries. Issues of ethics and bias in the use of AI are also addressed and indicators of the growth of knowledge in the field are discussed.

Interdisciplinary and international in scope, and bringing together leading academics and practitioners from across the globe, the book explores the applications of AI and big data in Rule of Law and development evaluation, identifies differences in the approaches used in the two fields, and how each could learn from the approaches used in the other, as well as differences in the AI-related issues addressed in industrialized nations compared to those addressed in Africa and Asia.

Artificial Intelligence and Big Data is an essential read for researchers, academics and students working in the fields of Rule of Law and Development, and researchers in institutions working on new applications in AI will all benefit from the book’s practical insights…(More)”.