Opens in a new tab
Explore our articles
View All Results
Share:

AI’s Accuracy Nears the Best Human Forecasters

Article by Samuel Greengard: “When business leaders, government officials, and others seek predictability in a highly unpredictable world, they turn to forecasts. Historically, the most accurate prognostications have come from a group of humans called superforecasters, who weigh probabilities and analyze questions before reaching conclusions.

These individuals—the top 2% of forecasters—perform about 30% better than professional intelligence analysts with access to classified information, according to research conducted by Barbara Mellers and Philip Tetlock. But now things are getting more complicated as AI forecasting enters the picture.

“AI has transformed trading in prediction markets and in financial markets more generally,” said Rajiv Sethi, a professor of economics at Barnard College, Columbia University, and an external professor at the Santa Fe Institute. “Now there’s a question of whether superforecasters will become obsolete due to the AI revolution.”

According to ForecastBench, a benchmarking tool that scores AI models against human forecasters, model accuracy has climbed steadily over the past two years, reaching near parity with superforecasters. Yet parity isn’t judgment, and a growing number of observers are concerned that profit incentives and a lack of visibility into how prediction systems function could invite trouble.

A growing question is whether it’s acceptable to trade explanation for accuracy. “We’re not yet at a point where anyone is willing to take the best forecasting models and rely on them for high-stakes decisions,” said Ezra Karger, a senior economist at the Federal Reserve Bank of Chicago who leads the team that built ForecastBench…(More)”.

Share
How to contribute:

Did you come across – or create – a compelling project/report/book/app at the leading edge of innovation in governance?

Share it with us at info@thelivinglib.org so that we can add it to the Collection!

About the Curator

Get the latest news right in your inbox

Subscribe to curated findings and actionable knowledge from The Living Library, delivered to your inbox every Friday

Related articles

Get the latest news right in your inbox

Subscribe to curated findings and actionable knowledge from The Living Library, delivered to your inbox every Friday