The rise and fall of peer review

Curated on Posted on by Stefaan Verhulst

Blog by Adam Mastroianni: “For the last 60 years or so, science has been running an experiment on itself. The experimental design wasn’t great; there was no randomization and no control group. Nobody was in charge, exactly, and nobody was really taking consistent measurements. And yet it was the most massive experiment ever run, and it included every scientist on Earth.

Most of those folks didn’t even realize they were in an experiment. Many of them, including me, weren’t born when the experiment started. If we had noticed what was going on, maybe we would have demanded a basic level of scientific rigor. Maybe nobody objected because the hypothesis seemed so obviously true: science will be better off if we have someone check every paper and reject the ones that don’t pass muster. They called it “peer review.”

This was a massive change. From antiquity to modernity, scientists wrote letters and circulated monographs, and the main barriers stopping them from communicating their findings were the cost of paper, postage, or a printing press, or on rare occasions, the cost of a visit from the Catholic Church. Scientific journals appeared in the 1600s, but they operated more like magazines or newsletters, and their processes of picking articles ranged from “we print whatever we get” to “the editor asks his friend what he thinks” to “the whole society votes.” Sometimes journals couldn’t get enough papers to publish, so editors had to go around begging their friends to submit manuscripts, or fill the space themselves. Scientific publishing remained a hodgepodge for centuries.

(Only one of Einstein’s papers was ever peer-reviewed, by the way, and he was so surprised and upset that he published his paper in a different journal instead.)

That all changed after World War II. Governments poured funding into research, and they convened “peer reviewers” to ensure they weren’t wasting their money on foolish proposals. That funding turned into a deluge of papers, and journals that previously struggled to fill their pages now struggled to pick which articles to print. Reviewing papers before publication, which was “quite rare” until the 1960s, became much more common. Then it became universal.

Now pretty much every journal uses outside experts to vet papers, and papers that don’t please reviewers get rejected. You can still write to your friends about your findings, but hiring committees and grant agencies act as if the only science that exists is the stuff published in peer-reviewed journals. This is the grand experiment we’ve been running for six decades.

The results are in. It failed…(More)”.

A catalyst for community-wide action on sustainable development

Curated on Posted on by Stefaan Verhulst

Article by Communities around the world are increasingly recognizing that breaking down silos and leveraging shared resources and interdependencies across economic, social, and environmental issues can help accelerate progress on multiple issues simultaneously. As a framework for organizing local development priorities, the world’s 17 Sustainable Development Goals (SDGs) uniquely combine a need for broad technical expertise with an opportunity to synergize across domains—all while adhering to the principle of leaving no one behind. For local leaders attempting to tackle intersecting issues using the SDGs, one underpinning question is how to support new forms of collaboration to maximize impact and progress?

In early May, over 100 people across the East Central Florida (ECF) region in the U.S. participated in “Partnership for the Goals: Creating a Resilient and Thriving Community,” a two-day multi-stakeholder convening spearheaded by a team of local leaders from the East Central Florida Regional Resilience Collaborative (ECFR2C), the Central Florida Foundation, the City of Orlando, Florida for Good, Orange County, and the University of Central Florida. The convening grew out of a multi-year resilience planning process that leveraged the SDGs as a framework for tackling local economic, social, and environmental priorities all at once.

To move from community-wide planning to community-wide action, the organizers experimented with a 17 Rooms process—a new approach to accelerating collaborative action for the SDGs pioneered by the Center for Sustainable Development at Brookings and The Rockefeller Foundation. We collaborated with the ECF local organizing team and, in the process, spotted a range of more broadly relevant insights that we describe here…(More)”.

Which Connections Really Help You Find a Job?

Curated on Posted on by Stefaan Verhulst

Article by Iavor Bojinov, Karthik Rajkumar, Guillaume Saint-Jacques, Erik Brynjolfsson, and Sinan Aral: “Whom should you connect with the next time you’re looking for a job? To answer this question, we analyzed data from multiple large-scale randomized experiments involving 20 million people to measure how different types of connections impact job mobility. Our results, published recently in Science Magazine, show that your strongest ties — namely your connections to immediate coworkers, close friends, and family — were actually the least helpful for finding new opportunities and securing a job. You’ll have better luck with your weak ties: the more infrequent, arm’s-length relationships with acquaintances.

To be more specific, the ties that are most helpful for finding new jobs tend to be moderately weak: They strike a balance between exposing you to new social circles and information and having enough familiarity and overlapping interests so that the information is useful. Our findings uncovered the relationship between the strength of the connection (as measured by the number of mutual connections prior to connecting) and the likelihood that a job seeker transitions to a new role within the organization of a connection.The observation that weak ties are more beneficial for finding a job is not new. Sociologist Mark Granovetter first laid out this idea in a seminal 1973 paper that described how a person’s network affects their job prospects. Since then, the theory, known as the “strength of weak ties,” has become one of the most influential in the social sciences — underpinning network theories of information diffusion, industry structure, and human cooperation….(More)”.

Research Methods in Deliberative Democracy

Curated on Posted on by Stefaan Verhulst

Book edited by Selen A. Ercan et al: “… brings together a wide range of methods used in the study of deliberative democracy. It offers thirty-one different methods that scholars use for theorizing, measuring, exploring, or applying deliberative democracy. Each chapter presents one method by explaining its utility in deliberative democracy research and providing guidance on its application by drawing on examples from previous studies. The book hopes to inspire scholars to undertake methodologically robust, intellectually creative, and politically relevant research. It fills a significant gap in a rapidly growing field of research by assembling diverse methods and thereby expanding the range of methodological choices available to students, scholars, and practitioners of deliberative democracy…(More)”.

Learnings on the Importance of Youth Engagement

Curated on Posted on by Stefaan Verhulst

Blog by  Anna Ibru and Dane Gambrell at The GovLab: “…In recent years, public institutions around the world are piloting new youth engagement initiatives like Creamos that tap the expertise and experiences of young people to develop projects, programs, and policies and address complex social challenges within communities. 

To learn from and scale best practices from international models of youth engagement, The GovLab has develop case studies about three path breaking initiatives: Nuortenbudjetti, Helsinki’s participatory budgeting initiative for youth; Forum Jove BCN, Barcelona’s youth led citizens’ assembly; and Creamos, an open innovation and coaching program for young social innovators in Chile. For government decision makers and institutions who are looking to engage and empower young people to get involved in their communities, develop real-world solutions, and strengthen democracy, these examples describe these initiatives and their outcomes along with guidance on how to design and replicate such projects in your community. Young people are still a widely untapped resource who are too-often left out in policy and program design. The United Nations affirms that it is impossible to meet the UN SDGs by 2030 without active participation of the 1.8 billion youth in the world. Government decision makers and institutions must capitalize on the opportunity to engage and empower young people. The successes of Nuortenbudjetti, Forum Jove BCN, and Creamos provide a roadmap for policymakers looking to engage in this space….(More)” See also:  Nuortenbudjetti: Helsinki’s Youth Budget; Creamos: Co-creating youth-led social innovation projects in Chile and Forum Jove BCN: Barcelona’s Youth Forum.

Screen Shot 2022 12 06 At 1.36.48 Pm

Can citizen deliberation address the climate crisis? Not if it is disconnected from politics and policymaking

Curated on Posted on by Stefaan Verhulst

Blog by John Boswell, Rikki Dean and Graham Smith: “..Modelled on the deliberative democratic ideal, much of the attention on climate assemblies focuses on their internal features. The emphasis is on their novelty in providing respite from the partisan bickering of politics-as-usual, instead creating space for the respectful free and fair exchange of reasons.

On these grounds, the Global Citizens’ Assembly in 2021 and experimental ‘wave’ of climate assemblies across European countries are promising. Participating citizens have demonstrated they can grapple with complex information, deliberate respectfully, and come to a well thought-through set of recommendations that are – every time – more progressive than current climate policies.

But, before we get carried away with this enthusiasm, it is important to focus on a fundamental point usually glossed over. Assemblies are too often talked about in magical terms, as if by their moral weight alone citizen recommendations will win the day through the forceless force of their arguments. But this expectation is naive.

Designing for impact requires much more attention to the nitty-gritty of how policy actually gets made. That means taking seriously the technical uncertainties and complexities associated with policy interventions, and confronting the political challenges and trade-offs required in balancing priorities in the shadow of powerful interests.

In a recent study, we have examined the first six national climate assemblies – in Ireland, France, the UK, Scotland, Germany and Denmark – to see how they tried to achieve impact. Our novel approach is to take the focus away from their (very similar) ‘internal design characteristics’ – such as random selection – and instead put it on their ‘integrative design characteristics’…(More)”.

Citizen assemblies and the challenges of democratic equality

Curated on Posted on by Stefaan Verhulst

Article by Annabelle Lever: “…Creating a citizens’ assembly that truly reflects society as a whole isn’t so simple, however. In particular, only a very small percentage of those invited to participate actually agree to do so. According to a 2017 study published European Journal of Political Research, the precise percentage depends on how large, complex and time-consuming the process is likely to be. It ranges from 4% for larger, more onerous assemblies to 30% in a couple of exceptional cases, and averaging out at 15% across all countries and all forms of assembly. As a consequence, the formal equality of opportunity that unweighted lotteries promise tends to result in assemblies skewed to the socially advantaged, the partisan, and those most confident in their practical and cognitive abilities, whatever the reality.

To create an assembly that is more descriptively representative of the population – or one that looks more like us – several approaches are used. One is to have an initial phase of unweighted selection followed by a second phase that uses weighted lotteries. Another is to use stratified sampling or forms of stratification from the beginning.

For the Climate Assembly UK, organisers sent out 20% of its 30,000 letters of invitation to people randomly selected from the lowest-income postcodes, and then used random stratified sampling by computer to select 110 participants from all the people who were over 16 and free on the relevant dates.

Because citizen assemblies are very small compared to the population as a whole – France’s Convention for the Climate was made up of just 150 people – the descriptively representative character of the assembly can occur on only a few dimensions. Organisers must therefore decide what population characteristics the assembly should embody and in what proportion. Randomisation thus does not preclude difficult moral, political and scientific choices about the assembly to be constructed, any more than it precludes voluntariness or self-selection…(More)”.

Institutions, Experts & the Loss of Trust

Curated on Posted on by Stefaan Verhulst

Essay by Henry E. Brady and Kay Lehman Schlozman: “Institutions are critical to our personal and societal well-being. They develop and disseminate knowledge, enforce the law, keep us healthy, shape labor relations, and uphold social and religious norms. But institutions and the people who lead them cannot fulfill their missions if they have lost legitimacy in the eyes of the people they are meant to serve.

Americans’ distrust of Congress is long-standing. What is less well-documented is how partisan polarization now aligns with the growing distrust of institutions once thought of as nonpolitical. Refusals to follow public health guidance about COVID-19, calls to defund the police, the rejection of election results, and disbelief of the press highlight the growing polarization of trust. But can these relationships be broken? And how does the polarization of trust affect institutions’ ability to confront shared problems, like climate change, epidemics, and economic collapse?…(More)”.

The Future of Self-Governing, Thriving Democracies

Curated on Posted on by Stefaan Verhulst

Book by Brigitte Geissel: “This book offers a new approach for the future of democracy by advocating to give citizens the power to deliberate and to decide how to govern themselves.

Innovatively building on and integrating components of representative, deliberative and participatory theories of democracy with empirical findings, the book provides practices and procedures that support communities of all sizes to develop their own visions of democracy. It revitalizes and reinfuses the ‘democratic spirit’ going back to the roots of democracy as an endeavor by, with and for the people, and should inspire us in our search for the democracy we want to live in.

This book is of key interest to scholars and students in democracy, democratic innovations, deliberation, civic education and governance and further for policy-makers, civil society groups and activists. It encourages us to reshape democracy based on citizens’ perspectives, aspirations and preferences…(More)”.

The network science of collective intelligence

Curated on Posted on by Stefaan Verhulst

Article by Damon Centola: “In the last few years, breakthroughs in computational and experimental techniques have produced several key discoveries in the science of networks and human collective intelligence. This review presents the latest scientific findings from two key fields of research: collective problem-solving and the wisdom of the crowd. I demonstrate the core theoretical tensions separating these research traditions and show how recent findings offer a new synthesis for understanding how network dynamics alter collective intelligence, both positively and negatively. I conclude by highlighting current theoretical problems at the forefront of research on networked collective intelligence, as well as vital public policy challenges that require new research efforts…(More)”.