New AI Knows When to Trust Itself — And When to Stay Quiet
Turns out asking an AI how sure it is tells you almost nothing useful.
Imagine you're trying to guess whether a company will miss its earnings, or who wins an election. You might have betting-market odds, a poll, a statistical model — and now, a chatbot's opinion. The hard question isn't whether the AI is smart. It's whether the AI adds anything the other sources don't already tell you. That's the puzzle this paper tackles, and the answer is: usually only in some topics, and only sometimes.
The team's fix is a "competence gate" — think of it as a volume dial for the AI. Instead of trusting the model equally everywhere, the system looks at how the AI performed on similar questions in the past and turns its volume up or down accordingly. When there's not enough history to be sure, it falls back toward a cautious average. Tested across 2,357 real yes/no questions and five language models, the dial improved accuracy from a score of 0.0771 to 0.0732 (lower is better) — a small but consistent gain over simpler "just blend everything together" approaches.
Here's the honest part. On the official forecast market questions, where real money is already being bet, the AI gave no meaningful improvement. The system figured that out and largely deferred to the market. And when the researchers asked four Qwen models to state how confident they felt, that self-reported confidence did not reliably reveal when the model was beating the market. What did work was the model's measured track record. In other words: an AI's swagger is not evidence.
For everyday life, this points toward a future where AI tools come with a résumé rather than a personality — measured, topic by topic, and quietly silent where they have no edge. That matters if you're using AI to make decisions about money, hiring, or planning. The useful question to ask any AI tool isn't "how sure are you?" It's "how have you done on questions like this before?"
- The team tested five AI models on 2,357 real yes/no questions, mixing AI guesses with markets, polls, and statistical forecasts.
- Accuracy improved modestly — a score of 0.0771 to 0.0732, where lower is better — beating simpler 'just average them' methods.
- Where betting markets already existed, the AI added nothing, and the system learned to step aside rather than override them.
- Asking AI models how confident they felt did not reveal when they were actually right; their past track record did.
Why It Matters
It points toward AI that steps aside when it has no edge — and knows when its guesses are worth money.