Free AI on Your Phone Just Got a Speed Boost From Qualcomm

llama.cpp is free software that lets you run an AI chatbot on your own gadget instead of paying a company every time you ask it something. On 25 September, the project released update b11189, which adds new GPU "kernels" — small helper programs that tell a graphics chip how to do AI math. These new ones are written specifically for Qualcomm's Adreno graphics chips, the kind inside many Android phones and some Windows laptops.

Why should you care? Because when AI runs on your own device, nothing you type leaves it. No subscription, no internet needed, no company reading your questions. The trade-off has always been speed: a phone is far weaker than a data centre full of servers, so squeezing more performance out of phone chips is exactly what makes local AI usable rather than a novelty. This update is that kind of unglamorous but important plumbing.

A quick translation of the jargon. The AI models involved are "quantized" — compressed to roughly a quarter of their original size so they fit in phone memory without becoming gibberish. "dp4a" is a chip instruction that does several small math problems in one go, like a checkout scanning four items at once. The update also fixes how data is rearranged before those calculations. It was co-written by Li He, an engineer at Qualcomm, the company that makes Snapdragon chips.

The honest catch: this is a developer release note, not an app you download from a store. It contains no published speed measurements, so nobody outside the project knows yet how much faster anything actually feels. It only helps devices with Qualcomm Adreno graphics — if you own an iPhone, a Mac, or a laptop with Nvidia or AMD graphics, nothing changes for you. And casual users will only feel the benefit once the apps built on top of llama.cpp ship their own updates.