Article by Samuel Greengard: “When business leaders, government officials, and others seek predictability in a highly unpredictable world, they turn to forecasts. Historically, the most accurate prognostications have come from a group of humans called superforecasters, who weigh probabilities and analyze questions before reaching conclusions.
These individuals—the top 2% of forecasters—perform about 30% better than professional intelligence analysts with access to classified information, according to research conducted by Barbara Mellers and Philip Tetlock. But now things are getting more complicated as AI forecasting enters the picture.
“AI has transformed trading in prediction markets and in financial markets more generally,” said Rajiv Sethi, a professor of economics at Barnard College, Columbia University, and an external professor at the Santa Fe Institute. “Now there’s a question of whether superforecasters will become obsolete due to the AI revolution.”
According to ForecastBench, a benchmarking tool that scores AI models against human forecasters, model accuracy has climbed steadily over the past two years, reaching near parity with superforecasters. Yet parity isn’t judgment, and a growing number of observers are concerned that profit incentives and a lack of visibility into how prediction systems function could invite trouble.
A growing question is whether it’s acceptable to trade explanation for accuracy. “We’re not yet at a point where anyone is willing to take the best forecasting models and rely on them for high-stakes decisions,” said Ezra Karger, a senior economist at the Federal Reserve Bank of Chicago who leads the team that built ForecastBench…(More)”.