llama.cpp Update Makes AI Run Faster on Your Phone

There's a free piece of software called llama.cpp that lets you run AI models — like chatbots or writing tools — directly on your own computer or phone, instead of sending your questions to a company's cloud server. This week, its developers released a new update (codenamed b10664) that makes AI run better on phones with Qualcomm processors, which are found in many Android devices.

The update adds special, hand-tuned instructions for two math operations that AI models constantly use: absolute value and logarithm. These may sound simple, but on a phone's dedicated AI chip (called the Hexagon processor), doing them efficiently can noticeably speed things up. Think of it like upgrading from a bicycle to a geared bike — same rider, but much smoother ride uphill.

What does this mean for you? If you use apps that run AI locally on your phone, you could see faster responses, lower battery drain, and less heat. Even apps that currently rely on the cloud could start moving more work to your device, which is a big win for privacy. You're no longer sending sensitive questions or documents to someone else's server.

The catch: this update only helps phones with the right Qualcomm chip. If your phone is older or uses a different processor, you won't feel any difference. But it's another step toward a future where powerful AI runs on your pocket device — anytime, anywhere, with no internet needed.