How to improve economic forecasting

Curated on Posted on by Stefaan Verhulst

Article by Nicholas Gruen: “Today’s four-day weather forecasts are as accurate as one-day forecasts were 30 years ago. Economic forecasts, on the other hand, aren’t noticeably better. Former Federal Reserve chair Ben Bernanke should ponder this in his forthcoming review of the Bank of England’s forecasting.

There’s growing evidence that we can improve. But myopia and complacency get in the way. Myopia is an issue because economists think technical expertise is the essence of good forecasting when, actually, two things matter more: forecasters’ understanding of the limits of their expertise and their judgment in handling those limits.

Enter Philip Tetlock, whose 2005 book on geopolitical forecasting showed how little experts added to forecasting done by informed non-experts. To compare forecasts between the two groups, he forced participants to drop their vague weasel words — “probably”, “can’t be ruled out” — and specify exactly what they were forecasting and with what probability. 

That started sorting the sheep from the goats. The simple “point forecasts” provided by economists — such as “growth will be 3.0 per cent” — are doubly unhelpful in this regard. They’re silent about what success looks like. If I have forecast 3.0 per cent growth and actual growth comes in at 3.2 per cent — did I succeed or fail? Such predictions also don’t tell us how confident the forecaster is.

By contrast, “a 70 per cent chance of rain” specifies a clear event with a precise estimation of the weather forecaster’s confidence. Having rigorously specified the rules of the game, Tetlock has since shown how what he calls “superforecasting” is possible and how diverse teams of superforecasters do even better. 

What qualities does Tetlock see in superforecasters? As well as mastering necessary formal techniques, they’re open-minded, careful, curious and self-critical — in other words, they’re not complacent. Aware, like Socrates, of how little they know, they’re constantly seeking to learn — from unfolding events and from colleagues…(More)”.

Rethinking the Role of Nudge in Public Policy

Curated on Posted on by Stefaan Verhulst

Paper by Sema Müge Özdemiray: “The view of achieving the desired results in public policies depends on steering individuals, with decisions and actions incompatible with rationality, in a predictable way has pushed policymakers to collaborate with psychology methods and theories. Accordingly, in the recent policy design of public authorities, there is an increasing interest in the nudge approach, which is considered a less costly, more liberal, more citizen-focused alternative to traditional policy instruments. Nudging, which has produced effective solutions for different social problems, has also brought with it many criticisms. These criticisms have led to questioning alternative and advanced new policy tools in the field of behavioral public policy. In this study, the “nudge-plus” approach is discussed as one of these policy tools, which was put forward by Peter John and Gerry Stoker and which argues that the criticisms directed to nudge can be overcome by incorporating a citizen-oriented perspective into the nudge approach. This study aims to draw attention to the prediction that the use of the nudge-plus method in public policy design can produce more effective results in line with today’s participatory and collaborative administration approach…(More)”.

The Crowdless Future? How Generative AI Is Shaping the Future of Human Crowdsourcing

Curated on Posted on by Stefaan Verhulst

Paper by Leonard Boussioux, Jacqueline Lane, Miaomiao Zhang, Vladimir Jacimovic, and Karim Lakhani: “This study investigates the capability of generative artificial intelligence (AI) in creating innovative business solutions compared to human crowdsourcing methods. We initiated a crowdsourcing challenge focused on sustainable, circular economy business opportunities. The challenge attracted a diverse range of solvers from a myriad of countries and industries. Simultaneously, we employed GPT-4 to generate AI solutions using three different prompt levels, each calibrated to simulate distinct human crowd and expert personas. 145 evaluators assessed a randomized selection of 10 out of 234 human and AI solutions, a total of 1,885 evaluator-solution pairs. Results showed comparable quality between human and AI-generated solutions. However, human ideas were perceived as more novel, whereas AI solutions delivered better environmental and financial value. We use natural language processing techniques on the rich solution text to show that although human solvers and GPT-4 cover a similar range of industries of application, human solutions exhibit greater semantic diversity. The connection between semantic diversity and novelty is stronger in human solutions, suggesting differences in how novelty is created by humans and AI or detected by human evaluators. This study illuminates the potential and limitations of both human and AI crowdsourcing to solve complex organizational problems and sets the groundwork for a possible integrative human-AI approach to problem-solving…(More)”.

It’s like jury duty, but for getting things done

Curated on Posted on by Stefaan Verhulst

Article by Hollie Russon Gilman and Amy Eisenstein: “Citizens’ assemblies have the potential to repair our broken politics…Imagine a democracy where people come together and their voices are heard and are translated directly into policy. Frontline workers, doctors, teachers, friends, and neighbors — young and old — are brought together in a random, representative sample to deliberate the most pressing issues facing our society. And they are compensated for their time.

The concept may sound radical. But we already use this method for jury duty. Why not try this widely accepted practice to tackle the deepest, most crucial, and most divisive issues facing our democracy?

The idea — known today as citizens’ assemblies — originated in ancient Athens. Instead of a top-down government, Athens used sortition — a system that was horizontal and distributive. The kleroterion, an allotment machine, randomly selected citizens to hold civic office, ensuring that the people had a direct say in their government’s dealings….(More)”.

The Worst People Run for Office. It’s Time for a Better Way.

Curated on Posted on by Stefaan Verhulst

Article by Adam Grant: “On the eve of the first debate of the 2024 presidential race, trust in government is rivaling historic lows. Officials have been working hard to safeguard elections and assure citizens of their integrity. But if we want public office to have integrity, we might be better off eliminating elections altogether.

If you think that sounds anti-democratic, think again. The ancient Greeks invented democracy, and in Athens many government officials were selected through sortition — a random lottery from a pool of candidates. In the United States, we already use a version of a lottery to select jurors. What if we did the same with mayors, governors, legislators, justices and even presidents?

People expect leaders chosen at random to be less effective than those picked systematically. But in multiple experiments led by the psychologist Alexander Haslam, the opposite held true. Groups actually made smarter decisions when leaders were chosen at random than when they were elected by a group or chosen based on leadership skill.

Why were randomly chosen leaders more effective? They led more democratically. “Systematically selected leaders can undermine group goals,” Dr. Haslam and his colleagues suggest, because they have a tendency to “assert their personal superiority.” When you’re anointed by the group, it can quickly go to your head: I’m the chosen one.

When you know you’re picked at random, you don’t experience enough power to be corrupted by it. Instead, you feel a heightened sense of responsibility: I did nothing to earn this, so I need to make sure I represent the group well. And in one of the Haslam experiments, when a leader was picked at random, members were more likely to stand by the group’s decisions.

Over the past year I’ve floated the idea of sortition with a number of current members of Congress. Their immediate concern is ability: How do we make sure that citizens chosen randomly are capable of governing?

In ancient Athens, people had a choice about whether to participate in the lottery. They also had to pass an examination of their capacity to exercise public rights and duties. In America, imagine that anyone who wants to enter the pool has to pass a civics test — the same standard as immigrants applying for citizenship. We might wind up with leaders who understand the Constitution…(More)”.

Wikipedia’s Moment of Truth

Curated on Posted on by Stefaan Verhulst

Article by Jon Gertner at the New York Times: “In early 2021, a Wikipedia editor peered into the future and saw what looked like a funnel cloud on the horizon: the rise of GPT-3, a precursor to the new chatbots from OpenAI. When this editor — a prolific Wikipedian who goes by the handle Barkeep49 on the site — gave the new technology a try, he could see that it was untrustworthy. The bot would readily mix fictional elements (a false name, a false academic citation) into otherwise factual and coherent answers. But he had no doubts about its potential. “I think A.I.’s day of writing a high-quality encyclopedia is coming sooner rather than later,” he wrote in “Death of Wikipedia,” an essay that he posted under his handle on Wikipedia itself. He speculated that a computerized model could, in time, displace his beloved website and its human editors, just as Wikipedia had supplanted the Encyclopaedia Britannica, which in 2012 announced it was discontinuing its print publication.

Recently, when I asked this editor — he asked me to withhold his name because Wikipedia editors can be the targets of abuse — if he still worried about his encyclopedia’s fate, he told me that the newer versions made him more convinced that ChatGPT was a threat. “It wouldn’t surprise me if things are fine for the next three years,” he said of Wikipedia, “and then, all of a sudden, in Year 4 or 5, things drop off a cliff.”..(More)”.

Innovation Can Reboot American Democracy

Curated on Posted on by Stefaan Verhulst

Blog by Suzette Brooks Masters: “A thriving multiracial pluralist democracy is an aspiration that many people share for America. Far from being inevitable, the path to such a future is uncertain.

To stretch how we think about American democracy’s future iterations and begin to imagine the contours of the new, we need to learn from what’s emergent. So I’m going to take you on a whirlwind tour of some experiments taking place here and abroad that are the bright spots illuminating possible futures ahead.

My comments are informed by a research report I wrote last year called Imagining Better Futures for American Democracy. I interviewed dozens of visionaries in a range of fields and with diverse perspectives about the future of our democracy and the role positive visioning and futures thinking could play in reinvigorating it.

As I discuss these bright spots, I want to emphasize that what is most certain now is the accelerating and destabilizing change we are experiencing. It’s critical therefore to develop systems, institutions, norms and mindsets to navigate that change boldly and responsibly, not pretend that tomorrow will continue to look like today.

Yet when paradigms shift, as they inevitably do and I would argue are right now, that’s a messy and confusing time that can cause lots of anxiety and disorientation. During these critical periods of transition, we must set aside or ‘hospice” some assumptions, mindsets, practices, and institutions, while midwifing, or welcoming in, new ones.

This is difficult to do in the best of times but can be especially so when, collectively, we suffer from a lack of imagination and vision about what American democracy could and should become.

It’s not all our fault — inertia, fear, distrust, cynicism, diagnosis paralysis, polarization, exceptionalism, parochialism, and a pervasive, dystopian media environment are dragging us down. They create very strong headwinds weakening both our appetite and our ability to dream bigger and imagine better futures ahead.

However, focusing on and amplifying promising innovations can change that dysfunctional dynamic by inspiring us and providing blueprints to act upon when the time is right.

Below I discuss two main types of innovations in the political sphere: election-related structural reforms and governance reforms, including new forms of civic engagement and government decision-making…(More)”.

Inclusive Cyber Policy Making

Curated on Posted on by Stefaan Verhulst

Toolkit by Global Digital Partnership: “Marginalised perspectives, particularly from women and LGBTQ+ communities, are largely absent in current cyber norm discussions. This is a serious issue, as marginalised groups often face elevated and specific threats in cyberspace

Our bespoke toolkit provides policymakers and other stakeholders with a range of resources to address this lack of inclusion, including:

  • A how-to guide on developing an inclusive process to develop a cybernorm or implement existing agreed norms
  • An introduction to key terms and concepts relevant to inclusivity and cybernorms
  • Key questions for facilitating inclusive stakeholder mapping processes
  • A mapping of regional and global cybernorm processes…(More)”.

The Eyewitness Community Survey: An Engaging Citizen Science Tool to Capture Reliable Data while Improving Community Participants’ Environmental Health Knowledge and Attitudes

Curated on Posted on by Stefaan Verhulst

Paper by Melinda Butsch Kovacic: “Many youths and young adults have variable environmental health knowledge, limited understanding of their local environment’s impact on their health, and poor environmentally friendly behaviors. We sought to develop and test a tool to reliably capture data, increase environmental health knowledge, and engage youths as citizen scientists to examine and take action on their community’s challenges. The Eyewitness Community Survey (ECS) was developed through several iterations of co-design. Herein, we tested its performance. In Phase I, seven youths audited five 360° photographs. In Phase II, 27 participants works as pairs/trios and audited five locations, typically 7 days apart. Inter-rater and intra-rater reliability were determined. Changes in participants’ knowledge, attitudes, behaviors, and self-efficacy were surveyed. Feedback was obtained via focus groups. Intra-rater reliability was in the substantial/near-perfect range, with Phase II having greater consistency. Inter-rater reliability was high, with 42% and 63% of Phase I and II Kappa, respectively, in the substantial/near-perfect range. Knowledge scores improved after making observations (p ≤ 0.032). Participants (85%) reported the tool to be easy/very easy to use, with 70% willing to use it again. Thus, the ECS is a mutually beneficial citizen science tool that rigorously captures environmental data and provides engaging experiential learning opportunities…(More)”.

What types of health evidence persuade actors in a complex policy system?

Curated on Posted on by Stefaan Verhulst

Article by Geoff Bates, Sarah Ayres, Andrew Barnfield, and Charles Larkin: “Good quality urban environments can help to prevent non-communicable diseases such as cardiovascular diseases, mental health conditions and diabetes that account for three quarters of deaths globally (World Health Organisation, 2022). More commonly however, poor quality living conditions contribute to poor health and widening inequalities (Adlakha & John, 2022). Consequently, many public health advocates hope to convince and bring together the stakeholders who shape urban development to help create healthier places.

Evidence is one tool that can be used to convince these stakeholders from outside the health sector to think more about health outcomes. Most of the literature on the use of evidence in policy environments has focused on the public sector, such as politicians and civil servants (e.g., Crow & Jones, 2018). However, urban development decision-making processes involve many stakeholders across sectors with different needs and agendas (Black et al., 2021). While government sets policy and regulatory frameworks, private sector organisations such as property developers and investors drive urban development and strongly influence policy agendas.

In our article recently published in Policy & Politics, What types of evidence persuade actors in a complex policy system?, we explore the use of evidence to influence different groups across the urban development system to think more about health outcomes in their decisions…

The key findings of the research were that:

  1. Evidence-based narratives have wide appeal. Narratives based on real-world and lived experiences help stakeholders to form an emotional connection with evidence and are effective for drawing attention to health problems. Powerful outcomes such as child health and mortality data are particularly persuasive. This builds on literature promoting the use of storytelling approaches for public sector actors by demonstrating its applicability within the private and third sectors….(More)”