Your Phone Could Soon Run Powerful AI Without the Cloud

Right now, when you ask a chatbot a question, your words usually travel to a giant data center, get processed by expensive machines, and travel back. That costs money, burns energy, and means someone else holds your data. A team of researchers wants to flip that: run the AI right on your phone or laptop instead. Their new system, HA-NPU, is designed for the small, power-sipping AI chips (called NPUs, or neural processing units) already built into modern phones and laptops.

The AI models in question are "hybrid attention" models — think of them as chatbots that are smart but lightweight, built to remember context without gobbling up memory. Popular examples include the Qwen and Kimi series. The problem: when these models run on a phone's chip without special care, they choke. The chip wastes time shuffling data around in memory, so the speed advantage vanishes.

HA-NPU fixes that by rearranging the order of operations — like reorganizing a kitchen so ingredients are always within arm's reach instead of in a pantry down the hall. The team reports the fix makes the slowest part of the AI run up to 36 times faster and use roughly 36 times less energy. Overall, a full AI request finishes up to twice as fast. The researchers say they will release the code publicly.

Why should you care? If this kind of work catches on, future phones could run capable assistants offline — on a plane, in a basement, with no signal. Your conversations would stay on your device, your battery would last longer, and AI features might stop being a monthly subscription. The catch: this is a research paper, not a product, and it only covers one piece of the puzzle. Getting a full chatbot working smoothly on every phone will take more work — and phone makers have to actually adopt it.