New Study: Small AI Can Spot Names in Your Notes Without the Cloud

Every time you highlight an address in an email or a bank flags a company name in a transaction, some AI is picking out the important words for you. This job is called named-entity recognition — think of it as a tireless highlighter that never gets bored. Today most of that work happens on distant servers, which means your text travels off your device. A researcher tested whether it can happen locally instead.

The study lined up nine systems, from a tiny 13-million-parameter tagger to an 8-billion-parameter chatbot running on the same machine. (Parameters are roughly a model's brain size.) On accuracy alone, a big local chatbot held its own. But the small specialised models matched it at one-ninth to one-twenty-fourth the size, answered in milliseconds rather than seconds, and produced zero malformed output. That last point matters: the smallest chatbot-style models broke on up to 27% of long inputs, returning garbled answers instead of clean lists.

There is a second, subtler warning. The models were asked how confident they felt about each name they found. Their confidence did track whether they were right, which is genuinely useful — you could set a threshold and only trust the sure answers. But they were reliably overconfident, claiming more certainty than they earned. In plain terms: don't take an AI's word for how sure it is.

The practical upshot is encouraging for privacy. If small models are good enough, your phone or laptop can do this work alone — no upload, no subscription, no waiting. You keep the data, you keep the speed. The trade-off is that accuracy still depends on the job: these systems led on clean news writing but slipped on messier text. Cheap and private is now realistic; flawless is not.