AI Math Breaks When You Write 50% Instead of 0.5

Here's a simple test you can run on any AI chatbot. Ask it "What is 0.5 times 80?" Then ask, "What is 50% of 80?" Same question. Same math. But a new study of five free, downloadable AI models found the answers often don't match — even though they should. The researcher generated thousands of math word problems and rewrote each one five different ways, swapping decimals for fractions, percentages, spelled-out words like "half," and converted units. In their original form, the models scored 97% to nearly 100% correct. Once the same numbers were written differently, accuracy slid to between 85% and 98%.

Some of that drop turned out to be a paperwork problem, not a thinking problem. The grading software simply didn't recognize one style of writing numbers — the scientific notation used for very large or small values, like 5×10⁻¹. The AI was answering correctly; the checker couldn't read the answer. The study's author notes this is a warning for anyone testing AI: a broken measuring stick can look exactly like a broken brain.

But a real error remained. One model, Mistral Small 4, scored only about 70% on questions involving unit conversions, and produced 265 wrong answers that were off by exactly one decimal place — the AI equivalent of misplacing a comma. That kind of mistake is dangerous precisely because it looks plausible. A dose calculation, a loan payment, or a budget figure that's ten times too big or too small won't jump out at you.

The study also tested a popular trick: ask the AI the same question several times in different formats and see if the answers agree. It didn't work. Rewording the numbers caught no more mistakes than simply rewording the question — and it triggered more false alarms, flagging correct answers as suspicious. The practical takeaway is unglamorous. AI is genuinely good at arithmetic, but the format you type it in changes the answer more than it should. If a number matters, write it plainly, ask twice, and check the decimal point yourself.