Free AI Tool llama.cpp Now Runs Faster on Ordinary Graphics Cards

llama.cpp is a free piece of software that lets you run AI models on your own laptop, desktop or phone instead of sending your questions to a company's servers. It's one of the most popular projects of its kind, with about 128,000 "stars" (a GitHub popularity marker) and thousands of volunteer contributors. Last week it published build b10899, a small pre-release update that speeds up a specific kind of math the AI does constantly.

The update targets Vulkan, a graphics standard used by AMD, Intel and Qualcomm chips — the hardware inside most Windows laptops, many Android phones, and even single-board computers like the Raspberry Pi. If your machine has an Nvidia card, none of this applies to you, because those use a different system called CUDA that's already well optimized. The work was specifically tuned for Qwen, a family of AI models that's popular because it runs well on modest hardware.

The honest catch: this is a small optimization, not a new capability. It makes certain calculations faster — particularly the tiny ones that dominate when you're chatting with a model one word at a time — but it won't turn a slow machine into a fast one. It's also a pre-release build, meaning it's an early test version rather than a polished release, and installing it takes a bit of technical comfort.

So what's the bigger picture? Projects like this are quietly making it possible to have a decent AI assistant that costs nothing per month, works without internet, and never sends your data anywhere. The progress comes in dozens of small updates like this one rather than dramatic announcements. If you're curious, it's worth watching — but there's no rush to install anything today.