New Study: More AI Thinking Isn't Always Better at Predicting the Future

When you hear about AI 'reasoning,' it usually sounds like more is better — the AI thinks harder, breaks the problem into steps, checks its work. But a new paper from researcher Yufeng Wang suggests that for forecasting — predicting whether something will or won't happen — extra thinking can actually be wasted effort. The study tested AI forecasting tools on yes-or-no prediction tasks, the kind used to guess election results, court rulings, or whether a product will launch on time. In each case, the AI could choose between different strategies: search the web for evidence, reason it out step by step, copy what betting markets already predict, or lean on a similar past event.

The key result: no single strategy won everywhere. For some kinds of questions, matching to a past historical example worked best. For others, simply copying the crowd — the way prediction markets or poll averages do — beat everything else. That's a big deal, because it means the fancy part of the AI isn't the hero. Knowing which source to trust is.

To test this, Wang built something called ReliabilityRoute. Think of it as a traffic controller for evidence: before answering, the AI checks signals like how much historical data exists, whether a market price is available, and how much the sources disagree. Then it picks one lane. One version of the rule was locked in using 2024 data; another quietly re-tunes itself as new outcomes come in. Both beat the researcher's other automated setups on a scoring system that penalizes wrong guesses — but only modestly.

The honest caveat is bigger than the win. Simple baselines — plain search and historical patterns — stayed highly competitive across 16 later rounds of testing. The gain from smart routing was real but small. So the takeaway isn't 'this AI cracks forecasting.' It's 'stop assuming more reasoning equals better answers.' If you use AI to make predictions at work, or you're deciding whether to trust an AI forecast, the useful lesson is to ask where the evidence came from before you ask how clever the model is.