Free AI Tool llama.cpp Now Runs Faster on AMD Graphics Cards

Most people meet AI through a website: you type, a company's server answers, and you pay monthly or per question. llama.cpp is the opposite approach. It's a free, open-source program (129,000 people have starred it on GitHub, a rough popularity vote) that downloads AI models onto your own laptop, desktop, or phone. Once installed, the AI works offline. No subscription, and no data leaves your house.

The update, labeled b11117, is a tuning pass for AMD graphics cards. Big AI models normally need enormous memory, so people use "quantization" — squeezing a model down to a fraction of its size, like compressing a photo file. The tradeoff is usually speed. This update rewrites two small internal routines so AMD cards handle the most heavily compressed models (called IQ2 and IQ3) more efficiently.

The release page also shows how far this idea has spread. There are builds for Windows, Mac, Linux, Android, and iOS; for Nvidia, AMD, Intel, and Qualcomm chips; even for Snapdragon phone processors and Huawei server hardware. That breadth matters. It means "AI on your own device" isn't a hobbyist fantasy anymore — it runs on hardware people already own, including phones.

The honest caveat: this is not a headline feature. There's no new chatbot and no new capability. It's a small speed improvement for one specific setup — an AMD graphics card plus heavily compressed models — and most users won't notice a difference. Installing llama.cpp still takes some comfort with command-line tools, though friendlier apps built on top of it exist. Think of it as a car getting a slightly better fuel injector, not a new engine.