Spring AI Forecasting Benchmark is Starting Over the last year and a half, Metaculus has been running a series of tournaments to benchmark AI's accuracy in predicting future events. These tournaments pit frontier models, bot developers, and a human baseline against each other to collectively push the boundaries of forecasting...
Main Takeaways Top Findings: * Pro forecasters significantly outperform bots: Our team of 10 Metaculus Pro Forecasters demonstrated superior performance compared to the top-10 bot team, with strong statistical significance (p = 0.00001) based on a one-sided t-test on Peer scores. * The bot team did not improve significantly in...
By Ben Wilson and John Bash from Metaculus Main Takeaways Top Findings: * Pro forecasters significantly outperform bots: Our team of 10 Metaculus Pro Forecasters demonstrated superior performance compared to the top-10 bot team, with strong statistical significance (p = 0.001) based on a one-sided t-test on spot peer scores....