Scientists Made AI Better at Numbers With a Game of 20 Questions
Fewer wrong figures in bills, reports, and forecasts — no retraining required.
AI chatbots are great with words and surprisingly shaky with numbers. Ask one for a precise figure — a price, a date, a percentage — and it often lands close but not exactly right. The reason is simple: these systems learned language, not arithmetic. They predict the next likely word rather than calculating, so digits come out sounding plausible instead of being correct.
The new method, called NumericJev, sidesteps that weakness instead of fixing it. Rather than requesting "give me the answer," it asks the AI a series of multiple-choice range questions, narrowing the possibilities step by step, like playing 20 Questions or guessing a number by halving it repeatedly. On a grid of 100 possible values, a ten-way version of the tree needs only two rounds of questions to land on the answer — and the original question stays in view the whole time, so the AI never forgets what it was asked.
The results are modest but meaningful. The method beat simply letting the AI pick from a list of candidate answers (a list containing the correct one) by 2.93 percentage points. Its average error was 1.84% versus 5.18% for the direct approach — roughly a threefold improvement. In a separate test on historical dates, it averaged 4.58% error, and when the correct value was supplied, its readout was flawless. Crucially, it requires no retraining, no access to the AI's internal workings, and the code is public.
The honest caveat: this is early academic work on narrow, controlled tests — number ranges and dates — not proof that your chatbot will suddenly nail your tax estimate. The error rate is lower, not zero. Still, for anyone who relies on AI to draft quotes, tables, or forecasts, a threefold drop in numerical sloppiness is the difference between a draft you check carefully and one you can actually use.
- Instead of blurting out a number, the AI answers a few 'is it bigger or smaller?' range questions until it narrows down the answer — like 20 Questions for digits.
- Average error dropped to 1.84% from 5.18% in their tests, and accuracy beat picking from a list by 2.93 percentage points.
- It works on existing AI models with zero retraining and the code is free to download, so companies can bolt it on rather than buy new models.
Why It Matters
Fewer wrong numbers from AI means less double-checking, fewer costly mistakes in quotes, invoices, and reports.