Free AI Software Gets Better at Running on Your Phone

There's a free piece of software called llama.cpp, and it's one of the most popular AI projects in the world — roughly 129,000 people have starred it on GitHub, the site where programmers share code. Its whole job is to let you run an AI model directly on your own computer or phone, instead of paying a company like OpenAI or Google every month to use theirs. Nothing you type leaves your device.

This week it shipped build b11045, a small update with one technical line: it added support for a math step called ROLL on Hexagon chips. Big AI models are really just thousands of tiny math steps stacked together, and "ops" are those individual steps. If a model needs a step the software doesn't know how to do, the model simply won't run. ROLL is one of those steps — essentially telling the software to shift a list of numbers along. Hexagon is the chip family inside many Android phones and the new Snapdragon laptops.

Why should you care? Because this is how AI slowly moves off the cloud and into your pocket. The release notes also show the software being built for Windows, Mac, Linux, Android and iOS, plus Nvidia chips, AMD chips, Vulkan graphics, Intel's OpenVINO, and even Huawei's openEuler and Ascend hardware. The pattern is clear: AI is being spread across every kind of device, including the one you're holding. Every little op like this one removes another obstacle.

The honest catch: nothing visibly changes for most people today. This is plumbing, not a new feature, and it only affects certain models on certain chips. Running AI locally still takes some technical patience and a fairly capable machine. But the direction of travel matters — free, private AI that works offline is getting steadily easier.