Free Tool That Runs AI on Your Own Phone Just Got Faster

Most AI chatbots you use today don't actually live on your computer. When you type a question, it travels to a company's data center, gets answered there, and comes back. That's why AI often needs a subscription, an internet connection, and a privacy policy you probably didn't read. The llama.cpp project exists to change that. It's free software, built by volunteers and contributors, that squeezes AI models down so they can run directly on your own laptop or phone. No monthly fee. No data leaving your house.

This week's update (a pre-release version called b11194) makes that local AI run faster on a specific kind of hardware: chips made by Qualcomm, the Snapdragon processors inside many Android phones and a growing number of Windows laptops. The change is quietly technical — it adds a new "kernel," which is basically a shortcut for how the computer does its math. By doing that math in smaller, compressed chunks (8-bit numbers instead of bigger ones), the software can work faster while using less memory. Think of it as packing your suitcase more tightly so you can carry the same trip with less effort.

Why should you care? Because the direction of travel is clear. Free, private, on-device AI is getting better every few weeks. Right now it's mostly a hobbyist's tool — the kind of thing you'd need patience and some tech confidence to install. But the same improvements that please tinkerers today tend to show up in polished consumer apps a year or two later. When that happens, you could have a genuinely useful AI assistant that never sends your emails, photos, or health questions to a server.

The catch: this is a "pre-release," meaning it's an early, testing version — not something you install like a normal app. It also only speeds up one narrow piece of hardware. So don't rush to download anything. Just know the free, private alternative to paid AI is quietly getting stronger.