Irreproducibility is not a sign of failure, but an inspiration for fresh ideas

Curated on Posted on by Stefaan Verhulst

Editorial at Nature: “Everyone’s talking about reproducibility — or at least they are in the biomedical and social sciences. The past decade has seen a growing recognition that results must be independently replicated before they can be accepted as true.

A focus on reproducibility is necessary in the physical sciences, too — an issue explored in this month’s Nature Physics, in which two metrologists argue that reproducibility should be viewed through a different lens. When results in the science of measurement cannot be reproduced, argue Martin Milton and Antonio Possolo, it’s a sign of the scientific method at work — and an opportunity to promote public awareness of the research process (M. J. T. Milton and A. Possolo Nature Phys. 26, 117–119; 2020)….

However, despite numerous experiments spanning three centuries, the precise value of G remains uncertain. The root of the uncertainty is not fully understood: it could be due to undiscovered errors in how the value is being measured; or it could indicate the need for new physics. One scenario being explored is that G could even vary over time, in which case scientists might have to revise their view that it has a fixed value.

If that were to happen — although physicists think it unlikely — it would be a good example of non-reproduced data being subjected to the scientific process: experimental results questioning a long-held theory, or pointing to the existence of another theory altogether.

Questions in biomedicine and in the social sciences do not reduce so cleanly to the determination of a fundamental constant of nature. Compared with metrology, experiments to reproduce results in fields such as cancer biology are likely to include many more sources of variability, which are fiendishly hard to control for.

But metrology reminds us that when researchers attempt to reproduce the results of experiments, they do so using a set of agreed — and highly precise — experimental standards, known in the measurement field as metrological traceability. It is this aspect, the authors contend, that helps to build trust and confidence in the research process….(More)”.

Community science: A typology and its implications for governance of social-ecological systems

Curated on Posted on by Stefaan Verhulst

Paper by Anthony Charles, Laura Loucks, Fikret Berkes, and Derek Armitage: “There is an increasing recognition globally of the role to be played by community science –scientific research and monitoring driven and controlled by local communities, and characterized by place-based knowledge, social learning, collective action and empowerment. In particular, community science can support social-ecological system transformation, and help in achieving better ‘fit’ between ecological systems and governance, at local and higher levels of decision making.

This paper draws on three examples of communities as central actors in the process of knowledge co-production to present a typology of community science, and to deduce a set of key principles/conditions for success.

The typology involves three social learning models in which the community acquires scientific knowledge by (1) engaging with external bodies, (2) drawing on internal volunteer scientific expertise, and/or (3) hiring (or contracting) in-house professional scientific expertise. All of these models share the key characteristic that the local community decides with whom they wish to engage, and in each case, social learning is fundamental. Some conditions that facilitate community science include: community-driven and community-control; flexibility across leadership models; connection to place and collective values; empowerment, agency and collective action; credible trust; local knowledge; and links to governance.

Community science is not a panacea for effecting change at the local level, and there is need for critical assessment of how it can help to fill governance gaps. Nevertheless, a considerable body of experience globally illustrates how local communities are drawing effectively on community science for better conservation and livelihood outcomes, in a manner compatible with broader trends toward ecosystem-based management and local stewardship….(More)”.

Re-imagining “Action Research” as a Tool for Social Innovation and Public Entrepreneurship

Curated on Posted on by Stefaan Verhulst

Stefaan G. Verhulst at The GovLab: “We live in challenging times. From climate change to economic inequality and forced migration, the difficulties confronting decision-makers are unprecedented in their variety, as well as in their complexity and urgency. Our standard policy toolkit seems stale and ineffective while existing governance institutions are increasingly outdated and distrusted.

To tackle today’s challenges, we need not only new solutions but new ways of arriving at solutions. In particular, we need fresh research methodologies that can provide actionable insights on 21st century conditions. Such methodologies would allow us to redesign how decisions are made, how public services are offered, and how complex problems are solved around the world. 

Rethinking research is a vast project, with multiple components. This new essay focuses on one particular area of research: action research. In the essay, I first explain what we mean by action research, and also explore some of its potential. I subsequently argue that, despite that potential, action research is often limited as a method because it remains embedded in past methodologies; I attempt to update both its theory and practice for the 21st century.

Although this article represents only a beginning, my broader goal is to re-imagine the role of action research for social innovation, and to develop an agenda that could provide for what Amar Bhide calls “practical knowledge” at all levels of decision making in a systematic, sustainable, and responsible manner.  (Full Essay Here).”

Imagining the Next Decade of Behavioral Science

Curated on Posted on by Stefaan Verhulst

Evan Nesterak at the Behavioral Scientist: “If you asked Richard Thaler in 2010, what he thought would become of the then very new field of behavioral science over the next decade, he would have been wrong, at least for the most part. Could he have predicted the expansion of behavioral economics research? Probably. The Nobel Prize? Maybe. The nearly 300 and counting behavioral teams in governments, businesses, and other organizations around the world? Not a chance. 

When we asked him a year and a half ago to sum up the 10 years since the publication of Nudge, he replied “Am I too old to just say OMG? … [Cass Sunstein and I] would never have anticipated one “nudge unit” much less 200….Every once in a while, one of us will send the other an email that amounts to just ‘wow.’”

As we closed last year (and the last decade), we put out a call to help us imagine the next decade of behavioral science. We asked you to share your hopes and fears, predictions and warnings, open questions and big ideas. 

We received over 120 submissions from behavioral scientists around the world. We picked the most thought-provoking submissions and curated them below.

We’ve organized the responses into three sections. The first section, Promises and Pitfalls, houses the responses about the field as whole—its identity, purpose, values. In that section, you’ll find authors challenging the field to be bolder. You’ll also find ideas to unite the field, which in its growth has felt for some like the “Wild West.” Ethical concerns are also top of mind. “Behavioral science has confronted ethical dilemmas before … but never before has the essence of the field been so squarely in the wheelhouse of corporate interests,” writes Phillip Goff.

In the second section, we’ve placed the ideas about specific domains. This includes “Technology: Nightmare or New Norm,” where Tania Ramos considers the possibility of a behaviorally optimized tech dystopia. In “The Future of Work,” Lazslo Bock imagines that well-timed, intelligent nudges will foster healthier company cultures, and Jon Jachomiwcz emphasizes the importance of passion in an economy increasingly dominated by A.I. In “Climate Change: Targeting Individuals and Systems” behavioral scientists grapple with how the field can pull its weight in this existential fight. You’ll also find sections on building better governments, health care at the digital frontier and final mile, and the next steps for education. 

The third and final section gets the most specific of all. Here you’ll find commentary on the opportunities (and obligations) for research and application. For instance, George Lowenstein suggests we pay more attention to attention—an increasingly scarce resource. Others, on the application side, ponder how behavioral science will influence the design of our neighborhoods and wonder what it will take to bring behavioral science into the courtroom. The section closes with ideas on the future of intervention design and ways we can continue to master our methods….(More)”.

The Experimenter’s Inventory: A catalogue of experiments for decision-makers and professionals

Curated on Posted on by Stefaan Verhulst

Report by the Alliance for Useful Evidence: “This inventory is about how you can use experiments to solve public and social problems. It aims to provide a framework for thinking about the choices available to a government, funder or delivery organisation that wants to experiment more effectively. We aim to simplify jargon and do some myth-busting on common misperceptions.
There are other guides on specific areas of experimentation – such as on randomised controlled trials – including many specialist technical textbooks. This is not a technical manual or guide about how to run experiments. Rather, this inventory is useful for anybody wanting a jargon-free overview of the types and uses of experiments. It is unique in its breadth – covering the whole landscape of social and policy experimentation, including prototyping, rapid cycle testing, quasi-experimental designs, and a range of different types of randomised trials. Experimentation can be a confusing landscape – and there are competing definitions about what constitutes an experiment among researchers, innovators and evaluation practitioners. We take a pragmatic approach, including different designs that are useful for public problem-solving, under our experimental umbrella. We cover ways of experimenting that are both qualitative and quantitative, and highlight what we can learn from different approaches….(More)”.

Information literacy in the age of algorithms

Curated on Posted on by Stefaan Verhulst

Report by Alison J. Head, Ph.D., Barbara Fister, Margy MacMillan: “…Three sets of questions guided this report’s inquiry:

  1. What is the nature of our current information environment, and how has it influenced how we access, evaluate, and create knowledge today? What do findings from a decade of PIL research tell us about the information skills and habits students will need for the future?
  2. How aware are current students of the algorithms that filter and shape the news and information they encounter daily? What
    concerns do they have about how automated decision-making systems may influence us, divide us, and deepen inequalities?
  3. What must higher education do to prepare students to understand the new media landscape so they will be able to participate in sharing and creating information responsibly in a changing and challenged world?
    To investigate these questions, we draw on qualitative data that PIL researchers collected from student focus groups and faculty interviews during fall 2019 at eight U.S. colleges and universities. Findings from a sample of 103 students and 37 professors reveal levels of awareness and concerns about the age of algorithms on college campuses. They are presented as research takeaways….(More)”.

Global problems need social science

Curated on Posted on by Stefaan Verhulst

Hetan Shah at Nature: “Without human insights, data and the hard sciences will not meet the challenges of the next decade…

I worry about the fact that the call prioritized science and technology over the humanities and social sciences. Governments must make sure they also tap into that expertise, or they will fail to tackle the challenges of this decade.

For example, we cannot improve global health if we take only a narrow medical view. Epidemics are social as well as biological phenomena. Anthropologists such as Melissa Leach at the University of Sussex in Brighton, UK, played an important part in curbing the West African Ebola epidemic with proposals to substitute risky burial rituals with safer ones, rather than trying to eliminate such rituals altogether.

Treatments for mental health have made insufficient progress. Advances will depend, in part, on a better understanding of how social context influences whether treatment succeeds. Similar arguments apply to the problem of antimicrobial resistance and antibiotic overuse.

Environmental issues are not just technical challenges that can be solved with a new invention. To tackle climate change we will need insight from psychology and sociology. Scientific and technological innovations are necessary, but enabling them to make an impact requires an understanding of how people adapt and change their behaviour. That will probably require new narratives — the purview of rhetoric, literature, philosophy and even theology.

Poverty and inequality call even more obviously for expertise beyond science and maths. The UK Economic and Social Research Council has recognized that poor productivity in the country is a big problem, and is investing up to £32.4 million (US$42 million) in a new Productivity Institute in an effort understand the causes and potential remedies.

Policy that touches on national and geographical identity also needs scholarly input. What is the rise of ‘Englishness’? How do we live together in a community of diverse races and religions? How is migration understood and experienced? These intangibles have real-world consequences, as demonstrated by the Brexit vote and ongoing discussions about whether the United Kingdom has a future as a united kingdom. It will take the work of historians, social psychologists and political scientists to help shed light on these questions. I could go on: fighting against misinformation; devising ethical frameworks for artificial intelligence. These are issues that cannot be tackled with better science alone….(More)”.

Technology Can't Fix Algorithmic Injustice

Curated on Posted on by Stefaan Verhulst

Annette Zimmerman, Elena Di Rosa and Hochan Kima at Boston Review: “A great deal of recent public debate about artificial intelligence has been driven by apocalyptic visions of the future. Humanity, we are told, is engaged in an existential struggle against its own creation. Such worries are fueled in large part by tech industry leaders and futurists, who anticipate systems so sophisticated that they can perform general tasks and operate autonomously, without human control. Stephen Hawking, Elon Musk, and Bill Gates have all publicly expressed their concerns about the advent of this kind of “strong” (or “general”) AI—and the associated existential risk that it may pose for humanity. In Hawking’s words, the development of strong AI “could spell the end of the human race.”

These are legitimate long-term worries. But they are not all we have to worry about, and placing them center stage distracts from ethical questions that AI is raising here and now. Some contend that strong AI may be only decades away, but this focus obscures the reality that “weak” (or “narrow”) AI is already reshaping existing social and political institutions. Algorithmic decision making and decision support systems are currently being deployed in many high-stakes domains, from criminal justice, law enforcement, and employment decisions to credit scoring, school assignment mechanisms, health care, and public benefits eligibility assessments. Never mind the far-off specter of doomsday; AI is already here, working behind the scenes of many of our social systems.

What responsibilities and obligations do we bear for AI’s social consequences in the present—not just in the distant future? To answer this question, we must resist the learned helplessness that has come to see AI development as inevitable. Instead, we should recognize that developing and deploying weak AI involves making consequential choices—choices that demand greater democratic oversight not just from AI developers and designers, but from all members of society….(More)”.

The Case for an Institutionally Owned Knowledge Infrastructure

Curated on Posted on by Stefaan Verhulst

Article by James W. Weis, Amy Brand and Joi Ito: “Science and technology are propelled forward by the sharing of knowledge. Yet despite their vital importance in today’s innovation-driven economy, our knowledge infrastructures have failed to scale with today’s rapid pace of research and discovery.

For example, academic journals, the dominant dissemination platforms of scientific knowledge, have not been able to take advantage of the linking, transparency, dynamic communication and decentralized authority and review that the internet enables. Many other knowledge-driven sectors, from journalism to law, suffer from a similar bottleneck — caused not by a lack of technological capacity, but rather by an inability to design and implement efficient, open and trustworthy mechanisms of information dissemination.

Fortunately, growing dissatisfaction with current knowledge-sharing infrastructures has led to a more nuanced understanding of the requisite features that such platforms must provide. With such an understanding, higher education institutions around the world can begin to recapture the control and increase the utility of the knowledge they produce.

When the World Wide Web emerged in the 1990s, an era of robust scholarship based on open sharing of scientific advancements appeared inevitable. The internet — initially a research network — promised a democratization of science, universal access to the academic literature and a new form of open publishing that supported the discovery and reuse of knowledge artifacts on a global scale. Unfortunately, however, that promise was never realized. Universities, researchers and funding agencies, for the most part, failed to organize and secure the investment needed to build scalable knowledge infrastructures, and publishing corporations moved in to solidify their position as the purveyors of knowledge.

In the subsequent decade, such publishers have consolidated their hold. By controlling the most prestigious journals, they have been able to charge for access — extracting billions of dollars in subscription fees while barring much of the world from the academic literature. Indeed, some of the world’s wealthiest academic institutions are no longer able or willing to pay the subscription costs required.

Further, by controlling many of the most prestigious journals, publishers have also been able to position themselves between the creation and consumption of research, and so wield enormous power over peer review and metrics of scientific impact. Thus, they are able to significantly influence academic reputation, hirings, promotions, career progressions and, ultimately, the direction of science itself.

But signs suggest that the bright future envisioned in the early days of the internet is still within reach. Increasing awareness of, and dissatisfaction with, the many bottlenecks that the commercial monopoly on research information has imposed are stimulating new strategies for developing the future’s knowledge infrastructures. One of the most promising is the shift toward infrastructures created and supported by academic institutions, the original creators of the information being shared, and nonprofit consortia like the Collaborative Knowledge Foundation and the Center for Open Science.

Those infrastructures should fully exploit the technological capabilities of the World Wide Web to accelerate discovery, encourage more research support and better structure and transmit knowledge. By aligning academic incentives with socially beneficial outcomes, such a system could enrich the public while also amplifying the technological and societal impact of investment in research and innovation.

We’ve outlined below the three areas in which a shift to an academically owned platforms would yield the highest impact.

  • Truly Open Access…
  • Meaningful Impact Metrics…
  • Trustworthy Peer Review….(More)”.

Philosophy Is a Public Service

Curated on Posted on by Stefaan Verhulst

Jonathon Keats at Nautilus: “…One of my primary techniques, adapted from philosophy, is to undertake large-scale thought experiments. In these experiments, I create alternative realities that provide perspectives on our own society, and provoke dialogue about who and what we want to become. Another of my techniques is to create philosophical instruments: tools and devices with which people can collectively investigate the places they inhabit.

The former technique is exemplified by Centuries of the Bristlecone, and other environmentally-calibrated clocks I’m developing in other cities, such as a timepiece modulated by the flow of rivers in Alaska, currently in planning at the Anchorage Museum.

The latter is exemplified by a project I initiated in Berlin in 2014, which I’ve now instigated in cities around the world. It’s a new kind of camera that produces a single exposure over a span of 100 years. People hide these cameras throughout their city, providing a means for the next generation to observe the decisions that citizens make about their urban environment: decisions about development and gentrification and sustainability. In a sense, these devices are intergenerational surveillance cameras. They prompt people to consider the long-term impact of their actions. They encourage people to act in ways that will change the picture to reflect what they want the next generation to see.

But the truth is that most of my projects—perhaps even the two I’ve just mentioned—combine techniques from philosophy and many other disciplines. In order to map out possible futures for society, especially while navigating the shifting terrain of climate change, the philosopher-explorer needs to be adaptable. And most likely you won’t have all the skills and tools you need. I believe that anyone can become a philosopher-explorer. The practice benefits from more practitioners. No particular abilities are needed, except a capacity for collaboration.

Ayear ago, I was invited by the Fraunhofer Institute for Building Physics to envision the city of the future. Through Fraunhofer’s artist-in-lab program, I had the opportunity to work with leading scientists and engineers, and to run computer simulations and physical experiments on state-of-the-art equipment in Stuttgart and Holzkirchen, Germany.

My starting point was to consider one of the most serious problems faced by cities today: sea level rise. Global sea levels are expected to increase by two-and-a-half meters by the end of the century, and as much as 15 meters in the next 300 years. With 11 percent of the world population living less than 10 meters above the current sea level, many cities will probably be submerged in the future: mega-cities including New York and Shanghai. One likely response is that people will migrate inland, seeking ever higher elevations.

The question I asked myself was this: Would it make more sense to stay put?…(More)”.