The Economist: “On September 5th yet another domain of human intelligence fell to the artificial variety. For the first time an AI won the seasonal Metaculus Cup, a proving ground for forecasters of a wide variety of events. Not only that, other AIs took the second and fifth places, leaving third and fourth for humans.
Hundreds of entrants had predicted the outcomes of questions that would be resolved by September. Would an American state or EU country restrict data-centre development? Would the hantavirus outbreak affect at least five people who were not passengers on MV Hondius? How much would Brent crude cost? The participants were scored on the distance of their prediction from the true answer. Those that were closest for longest won the greatest number of points.
The human participants in the competition shared a $5,000 cash prize in proportion to the number of points that they picked up. But this sum is actually less than the fifth-placer had achieved in the real world. By betting on questions on which its AI thinks the market is wrong, FutureSearch has seen a 6% increase since June on the initial $100,000 value of its portfolio on Kalshi, a popular prediction market. Even that, though, is chump change compared with the performance of one of the developers behind Preseen who, despite his firm failing to make the top five in the competition, had turned $35 into $1.94m over seven months, the sixth-best return in Kalshi’s history.
Computers have long been used to make forecasts in narrow fields, such as the weather. But AI forecasters, like those on Metaculus, are a different breed. Instead of being trained specifically on data regarding the questions they are predicting the answers to, they are built using the same large language models (LLMs) that underpin the rest of the AI boom.
These bots therefore read the news and make sense of data in much the same way that human forecasters manage—except they do so far more broadly and swiftly. This lets them explain the reasoning behind their forecasts explicitly, a useful trait when persuading decision-makers to take their bets seriously…(More)”.
