The Free AI Tool That Runs on Your Own Phone Just Got Faster

Most AI you use — ChatGPT, Gemini, Copilot — actually runs in giant data centers owned by big companies. You send your question over the internet, their computers do the thinking, and the answer comes back. You pay a subscription, and your words travel through someone else's machines. llama.cpp is the best-known tool for the opposite approach: running AI entirely on your own laptop or phone, for free, with nothing leaving the device.

On September 18, the team behind llama.cpp published a small update (version b11044) that improves how the software works on Qualcomm's Hexagon chips. Those chips sit inside a huge share of Android phones and the new wave of Snapdragon laptops. The technical change involves something called IM2COL, which is a way of rearranging image and audio data so an AI model can process it efficiently. In plain terms: the software got better at feeding pictures and sound into AI running on your phone's hardware, and doing it without wasting memory.

Why should you care about internal plumbing? Because local AI is only appealing if it's fast. A chatbot that takes 30 seconds to answer on your phone is useless; the same one that answers in two seconds replaces a cloud service. Every one of these unglamorous updates shaves a little more time off, which is how free, private, offline AI slowly becomes practical for normal people instead of just hobbyists. The project has 129,000 stars on GitHub — a rough popularity score — making it one of the most widely used pieces of free AI software in the world.

The honest caveat: this release contains no new features you can see or click. The improvements apply mainly to devices with Qualcomm chips, and they matter most for image- and audio-heavy AI, not plain text chat. If you use a local AI app today, you might get a modest speed bump after updating. If you don't, nothing changes for you yet. Think of it as road maintenance — boring, invisible, and exactly what makes the drive possible later.