Free Tool Now Runs AI Chatbots on Your Phone, No Internet Needed

A free software project called llama.cpp — think of it as a way to run AI chatbots on your own laptop or phone instead of renting them from a big company — just released a small update called b11175. The change adds support for a specific way of shrinking AI models so they fit comfortably on Qualcomm's Hexagon chip, the AI-focused processor built into many Android phones and some Windows laptops. Nothing dramatic happened, but every one of these small updates moves the same idea forward: AI that lives on your device.

Why should you care? Because almost every chatbot you use today works by sending your words to a giant data center somewhere. That costs money, needs a connection, and means your questions — medical, financial, personal — travel through someone else's computers. When AI runs locally, all of that stays with you. No subscription. No signal needed on a plane or in a basement. No company reading your chats.

The technical trick here is called 'quantization' — a fancy word for compressing an AI model, the way you'd shrink a giant photo file so it still looks good but takes up far less space. This update adds a new compression setting (called q5_k) tuned for that phone chip, which should mean slightly better quality or speed for the same amount of memory.

The catch: this is plumbing, not a product. There's no app to download, no button to press. You'd need to be fairly technical, or wait for the apps built on top of tools like this. Also, the AI you can run on a phone today is still much less capable than the biggest cloud models. So don't cancel your chatbot subscription yet — but do expect the gap to keep closing, quietly, one small update at a time.