Old-School AI Beats Big Chatbots at Sorting Text — for 100x Less
A 20-year-old method matched giant AI models while running up to 486x faster.
A team of research-computing scientists asked a question that worries a lot of IT budgets: now that AI chatbots exist, should we throw away the old, cheap methods for sorting text? Their test case was Naive Bayes — a decades-old technique that guesses what a piece of text is about by counting which words appear in it. They ran it head-to-head against large language models (the technology behind chatbots) from four different families, spanning a 37x range in size, up to a 1-trillion-parameter giant.
The results were striking. When there were no examples to learn from, the big models won — 98.0% versus 88.2% on one Amazon review task. But the authors warn that win may be fake: the models may have already seen that test data while training. On a cleaner sentiment test, the old method actually won, 81.7% to 73.0%. And once you have labeled examples — say 10,000 tagged documents — Naive Bayes hit 89.1% accuracy on news classification, statistically tied with the 27-billion-parameter model (89.0%) and ahead of a far larger 397-billion model (84.8%).
The real gap is speed and cost. The researchers measured that batched AI inference on a graphics chip is 40 to 486 times slower than Naive Bayes running on an ordinary processor — and it uses roughly 100 times more energy per item. That's not a small efficiency win; it's the difference between a modest electricity bill and a data-center budget. A fine-tuned smaller AI model reached 90.6% accuracy, slightly higher, but at a fraction of the throughput.
The catch is that the answer depends on the job. For sentiment tasks with no labeled data, the big AI models stay ahead no matter how much data you add. For topic sorting, the old method catches up at around 10,000 labeled examples. To make this practical, the team released a free tool that automatically picks the right approach based on your data and configurable thresholds.
- Naive Bayes, a text-sorting method from the 1990s, tied a 27-billion-parameter AI model at 89.1% vs 89.0% accuracy once labeled examples were available.
- Running the old method on a regular CPU was 40 to 486 times faster than running AI models on expensive graphics chips — with about 100x less energy per item.
- The old method only loses when you have zero labeled examples; big AI models still win on no-data sentiment tasks, and may score well partly because they already saw the test data.
Why It Matters
Your company may be paying huge AI bills for text-sorting jobs a laptop could handle.