AI Chatbots Waste a Third of Their Reading Effort, Study Finds

Every time you type a question into an AI chatbot, the first thing that happens is invisible. Before the AI reads a single word, a program called a tokeniser chops your text into small pieces — roughly word-sized chunks called tokens. Those chop points are hidden from you, but they shape everything that follows, including how much the AI company pays to run the model and, often, how much you pay to use it. Two researchers, Yuhao Du and Shunian Chen, set out to measure exactly how much those chop points cost.

The answer, published on the research site arXiv, is a lot. On English Wikipedia, the rules about where you are allowed to cut text increase the number of tokens needed by 28.3% to 37%. In everyday terms, the same paragraph takes up roughly a third more space than it really needs. Since AI services bill by the token — like a taxi charging by the mile — that is like paying for extra miles on a route you did not take. The researchers also built a checking tool so their numbers can be independently verified.

But here is the twist. Packing text more tightly does not automatically make the AI better at predicting what comes next. In tests across 12 languages, the setups that squeezed text smallest produced worse predictions than the ones that packed less efficiently. Compression and prediction, the authors found, want different things. So a company cannot simply flip a switch and get both cheaper and smarter — it has to choose.

There is a middle path. The team tested "boundary licences," a rule that lets just 10% of the AI's vocabulary cross the cut lines. That recovered 85% of the savings in English and 100% in Chinese — most of the benefit for a fraction of the disruption. Important caveat: this is a theoretical paper, not a product anyone has shipped. But it puts a number on a hidden cost that shows up in AI bills everywhere.