Can We Trust Social Science Yet?

Curated on Posted on by Stefaan Verhulst

Essay by Ryan Briggs: “Everyone likes the idea of evidence-based policy, but it’s hard to realize it when our most reputable social science journals are still publishing poor quality research.

Ideally, policy and program design is a straightforward process: a decision-maker faces a problem, turns to peer-reviewed literature, and selects interventions shown to work. In reality, that’s rarely how things unfold. The popularity of “evidence-based medicine” and other “evidence-based” topics highlights our desire for empirical approaches — but would the world actually improve if those in power consistently took social science  evidence seriously? It brings me no joy to tell you that, at present, I think the answer is usually “no.”

Given the current state of evidence production in the social sciences, I believe that many — perhaps most — attempts to use social scientific evidence to inform policy will not lead to better outcomes. This is not because of politics or the challenges of scaling small programs. The problem is more immediate. Much of social science research is of poor quality, and sorting the trustworthy work from bad work is difficult, costly, and time-consuming.

But it is necessary. If you were to randomly select an empirical paper published in the past decade — including any studies from the top journals in political science or economics — there is a high chance that its findings may be inaccurate. And not just off by a little: possibly two times as large, or even incorrectly signed. As an academic, this bothers me. I think it should bother you, too. So let me explain why this happens…(More)”.

Public AI White Paper – A Public Alternative to Private AI Dominance

Curated on Posted on by Stefaan Verhulst

White paper by the Bertelsmann Stiftung and Open Future: “Today, the most advanced AI systems are developed and controlled by a small number of private companies. These companies hold power not only over the models themselves but also over key resources such as computing infrastructure. This concentration of power poses not only economic risks but also significant democratic challenges.

The Public AI White Paper presents an alternative vision, outlining how open and public-interest approaches to AI can be developed and institutionalized. It advocates for a rebalancing of power within the AI ecosystem – with the goal of enabling societies to shape AI actively, rather than merely consume it…(More)”.

“R&D” Means Something Different on Capitol Hill

Curated on Posted on by Stefaan Verhulst

Article by Sheril Kirshenbaum: “My first morning as a scientist-turned-Senate-staffer began with a misunderstanding that would become a metaphor for my impending immersion into the complex world of policymaking. When my new colleagues mentioned “R&D,” I naively assumed they were discussing critical topics related to research and development. After 10 or so confused minutes, I realized they were referring to Republicans and Democrats—my first lesson in the distinctive language and unique dynamics of congressional work. The “R&D” at the center of their world was vastly different than that of mine.In the 20 years since, I’ve moved between academic science positions and working on science policy in the Senate, under both Republican and Democratic majorities. My goal during these two decades has remained the same—to promote evidence-based policymaking that advances science and serves the public, regardless of the political landscape. But the transition from scientist to staffer has transformed my understanding of why so many efforts by scientists to influence policy falter. Despite generations of scholarly research to understand how information informs political decisions, scientists and other academics consistently overlook a crucial part of the process: the role of congressional staffers.

The staff hierarchy shapes how scientific information flows to elected officials. Chiefs of staff manage office operations and serve as the member’s closest advisors. Legislative directors oversee all policy matters, while legislative assistants (LAs) handle specific issue portfolios. One or two LAs may be designated as the office “science people,” although they often lack formal scientific training. Committee staffers provide deeper expertise and institutional knowledge on topics within their jurisdiction. In this ecosystem, few dedicated science positions exist, and science-related topics are distributed among staff already juggling multiple responsibilities…(More)”

Playing for science: Designing science games

Curated on Posted on by Stefaan Verhulst

Paper by Claudio M Radaelli: “How can science have more impact on policy decisions? The P-Cube Project has approached this question by creating five pedagogical computer games based on missions given to a policy entrepreneur (the player) advocating for science-informed policy decisions. The player explores simplified strategies for policy change rooted in a small number of variables, thus making it possible to learn without a prior background in political science or public administration. The games evolved from the intuition that, instead of making additional efforts to explain science to decision-makers, we should directly empower would-be scientists (our primary audience for the games), post-graduates in public policy and administration, and activists for science. The two design principles of the games revolve around learning about how policy decisions are made (a learning-about-content principle) and reflection. Indeed, the presence of science in the policy process raises ethical and normative decisions, especially when we consider controversial strategies like civil disobedience and alliances with industry. To be on the side of science does not mean to be outside society and politics. I show the motivation, principles, scripts and pilots of the science games, reflecting on how they can be used and for what reasons…(More)”

Entering the Vortex

Curated on Posted on by Stefaan Verhulst

Essay by Nils Gilman: “A strange and unsettling weather pattern is forming over the landscape of scholarly research. For decades, the climate of academic inquiry was shaped by a prevailing high-pressure system, a consensus grounded in the vision articulated by Vannevar Bush in “Science: The Endless Frontier” (1945). That era was characterized by robust federal investment, a faith in the university as the engine of basic research, and a compact that traded public funding for scientific autonomy and the promise of long-term societal benefit. It was a climate conducive to the slow, deliberate, and often unpredictable growth of knowledge, nurtured by a diverse ecosystem of human researchers — the vital “seed stock” of intellectual discovery.

But that high-pressure system is collapsing. A brutal, unyielding cold front of academic defunding has swept across the nation, a consequence of shifting political priorities, populist resentment, and a calculated assault on the university as an institution perceived as hostile to certain political agendas. This is not merely a belt-tightening exercise; it is, for all intents and purposes, the dismantling of Vannevar Bush’s Compact, the end of the era of “big government”-funded Wissenschaft. Funding streams for basic research are dwindling, grant applications face increasingly long odds, and the financial precarity of academic careers deters the brightest minds. The human capital necessary for sustained, fundamental inquiry is beginning to wither.

Simultaneously, a warm, moisture-laden airmass is rapidly advancing: the astonishing rise of AI-based research tools. Powered by vast datasets and sophisticated algorithms, these tools promise to revolutionize every stage of the research process – from literature review and data analysis to hypothesis generation and the drafting of scholarly texts. As a recent New Yorker piece on AI and the humanities suggests, these AI engines can already generate deep research and coherent texts on virtually any subject, seemingly within moments. They offer the prospect of unprecedented efficiency, speed, and scale in the production of scholarly output.

The collision of these two epochal weather systems — the brutal cold front of academic defunding and the warm, expansive airmass of AI-based research tools — is creating an atmospheric instability unlike anything the world of scholarship has ever witnessed. Along the front where these forces meet, a series of powerful and unpredictable tornados are beginning to touch down, reshaping the terrain of knowledge production in real-time…(More)”.

Real-time prices, real results: comparing crowdsourcing, AI, and traditional data collection

Curated on Posted on by Stefaan Verhulst

Article by Julius Adewopo, Bo Andree, Zacharey Carmichael, Steve Penson, Kamwoo Lee: “Timely, high-quality food price data is essential for shock responsive decision-making. However, in many low- and middle-income countries, such data is often delayed, limited in geographic coverage, or unavailable due to operational constraints. Traditional price monitoring, which relies on structured surveys conducted by trained enumerators, is often constrained by challenges related to cost, frequency, and reach.

To help overcome these limitations, the World Bank launched the Real-Time Prices (RTP) data platform. This effort provides monthly price data using a machine learning framework. The models combine survey results with predictions derived from observations in nearby markets and related commodities. This approach helps fill gaps in local price data across a basket of goods, enabling real-time monitoring of inflation dynamics even when survey data is incomplete or irregular.

In parallel, new approaches—such as citizen-submitted (crowdsourced) data—are being explored to complement conventional data collection methods. These crowdsourced data were recently published in a Nature Scientific Data paper. While the adoption of these innovations is accelerating, maintaining trust requires rigorous validation.

A newly published study in PLOS compares the two emerging methods with the traditional, enumerator-led gold standard, providing  new evidence that both crowdsourced and AI-imputed prices can serve as credible, timely alternatives to traditional ground-truth data collection—especially in contexts where conventional methods face limitations…(More)”.

These Startups Are Building Advanced AI Models Without Data Centers

Curated on Posted on by Stefaan Verhulst

Article by Will Knight: “Researchers have trained a new kind of large language model (LLM) using GPUs dotted across the world and fed private as well as public data—a move that suggests that the dominant way of building artificial intelligence could be disrupted.

Article by Will Knight: “Flower AI and Vana, two startups pursuing unconventional approaches to building AI, worked together to create the new model, called Collective-1.

Flower created techniques that allow training to be spread across hundreds of computers connected over the internet. The company’s technology is already used by some firms to train AI models without needing to pool compute resources or data. Vana provided sources of data including private messages from X, Reddit, and Telegram.

Collective-1 is small by modern standards, with 7 billion parameters—values that combine to give the model its abilities—compared to hundreds of billions for today’s most advanced models, such as those that power programs like ChatGPT, Claude, and Gemini.

Nic Lane, a computer scientist at the University of Cambridge and cofounder of Flower AI, says that the distributed approach promises to scale far beyond the size of Collective-1. Lane adds that Flower AI is partway through training a model with 30 billion parameters using conventional data, and plans to train another model with 100 billion parameters—close to the size offered by industry leaders—later this year. “It could really change the way everyone thinks about AI, so we’re chasing this pretty hard,” Lane says. He says the startup is also incorporating images and audio into training to create multimodal models.

Distributed model-building could also unsettle the power dynamics that have shaped the AI industry…(More)”

In Uncertain Times, Get Curious

Curated on Posted on by Stefaan Verhulst

Chapter (and book) by Elizabeth Weingarten: “Questions flow from curiosity. If we want to live and love the questions of our lives—How to live a life of purpose? Who am I in the aftermath of a big change or transition? What kind of person do I want to become as I grow older?—we must first ask them into conscious existence.

Many people have written entire books defining and redefining curiosity. But for me, the most helpful definition comes from a philosophy professor, Perry Zurn, and a systems neuroscientist, Dani Bassett: “For too long—and still too often—curiosity has been oversimplified,” they write, typically “reduced to the simple act of raising a hand or voicing a question, especially from behind a desk or a podium. . . . Scholars generally boil it down to ‘information-seeking’ behavior or a ‘desire to know.’ But curiosity is more than a feeling and certainly more than an act. And curiosity is always more than a single move or a single question.”Curiosity works, they write, by “linking ideas, facts, perceptions, sensations and data points together.”It is complex, mutating, unpredictable, and transformational. It is, fundamentally, an act of connection, an act of creating relationships between ideas and people. Asking questions then, becoming curious, is not just about wanting to find the answer—it is also about our need to connect, with ourselves, with others, with the world.

And this, perhaps, is why our deeper questions are hardly ever satisfied by Google or by fast, easy answers from the people I refer to as the Charlatans of Certainty—the gurus, influencers, and “experts” peddling simple solutions to all the complex problems you face. This is also the reason there is no one-size-fits-all formula for cultivating curiosity—particularly the kind that allows us to live and love our questions, especially the questions that are hard to love, like “How can I live with chronic pain?” or “How do I extricate myself from a challenging relationship?” This kind of curiosity is a special flavor…(More)”. See also: Inquiry as Infrastructure: Defining Good Questions in the Age of Data and AI.

Mapping local knowledge supports science and stewardship

Curated on Posted on by Stefaan Verhulst

Paper by Sarah C. Risley, Melissa L. Britsch, Joshua S. Stoll & Heather M. Leslie: “Coastal marine social–ecological systems are experiencing rapid change. Yet, many coastal communities are challenged by incomplete data to inform collaborative research and stewardship. We investigated the role of participatory mapping of local knowledge in addressing these challenges. We used participatory mapping and semi-structured interviews to document local knowledge in two focal social–ecological systems in Maine, USA. By co-producing fine-scale characterizations of coastal marine social–ecological systems, highlighting local questions and needs, and generating locally relevant hypotheses on system change, our research demonstrates how participatory mapping and local knowledge can enhance decision-making capacity in collaborative research and stewardship. The results of this study directly informed a collaborative research project to document changes in multiple shellfish species, shellfish predators, and shellfish harvester behavior and other human activities. This research demonstrates that local knowledge can be a keystone component of collaborative social–ecological systems research and community-lead environmental stewardship…(More)”.

Inquiry as Infrastructure: Defining Good Questions in the Age of Data and AI

Curated on Posted on by Stefaan Verhulst

Paper by Stefaan Verhulst: “The most consequential failures in data-driven policymaking and AI deployment often stem not from poor models or inadequate datasets but from poorly framed questions. This paper centers question literacy as a critical yet underdeveloped competency in the data and policy landscape. Arguing for a “new science of questions,” it explores what constitutes a good question-one that is not only technically feasible but also ethically grounded, socially legitimate, and aligned with real-world needs. Drawing on insights from The GovLab’s 100 Questions Initiative, the paper develops a taxonomy of question types-descriptive, diagnostic, predictive, and prescriptive-and identifies five essential criteria for question quality: questions must be general yet concrete, co-designed with affected communities and domain experts, purpose-driven and ethically sound, grounded in data and technical realities, and capable of evolving through iterative refinement. The paper also outlines common pathologies of bad questions, such as vague formulation, biased framing, and solution-first thinking. Rather than treating questions as incidental to analysis, it argues for institutionalizing deliberate question design through tools like Q-Labs, question maturity models, and new professional roles for data stewards. Ultimately, the paper contends that the questions are infrastructures of meaning. What we ask shapes not only what data we collect or what models we build but also what values we uphold and what futures we make possible…(More)”.