Free AI Software llama.cpp Gets a Small Speed Boost for Android Phones

llama.cpp is one of the most popular pieces of free software in the AI world — roughly 129,000 people have "starred" it on GitHub, the site where programmers share code. Its whole point is unusual: it lets you run AI chatbots on hardware you already own, like a laptop or an Android phone, instead of renting time from companies such as OpenAI or Google. Nothing gets sent to a server. Nothing gets billed monthly.

The update released on 19 September, labelled b11055, is mainly a builder's release: fresh downloadable versions of the software for Macs, Windows PCs, Linux machines, iPhones and Android phones. The one genuinely new ingredient is support for something called GEGLU_QUICK on Qualcomm's Hexagon chips — the AI-capable sections inside many Android phones and some laptops. GEGLU is one of the many small maths steps a chatbot performs billions of times to produce a sentence. A quicker version of that step means less waiting, especially on battery-powered devices.

Why should you care about a maths tweak? Because the direction of travel is toward AI that lives on your device. That means working on a plane with no Wi-Fi, keeping private notes or health questions off someone else's servers, and skipping subscription fees that often run about $20 a month. Every small optimisation like this one nudges local AI from a hobbyist's toy toward something your phone can genuinely do well.

The catch is honesty about scale. This is not a new app you can grab from an app store; llama.cpp is a command-line tool, and most people need a technical friend to set it up. The quality of AI models that fit on a phone still trails the big cloud chatbots. And this particular change only helps certain Qualcomm chips. Useful progress — modest news.