Free AI Software Now Runs Faster on Your Own AMD Radeon Card

llama.cpp is free, open-source software that lets you download an AI model and run it on your own laptop or desktop. No account, no monthly bill, and nothing you type gets sent to a company's servers. Build b10877 is a minor update. Its one real change: better math settings for AMD's newer Radeon graphics cards (the RX 7000 series, called "RDNA3" in engineer speak), aimed specifically at so-called mixture-of-experts models.

Mixture of experts is easier to picture as a giant office building full of specialists. Instead of the whole building answering every question, only two or three relevant experts chime in. That trick is how companies make enormous AI models affordable to run. The catch is that routing work to the right expert quickly on a graphics card is fiddly. This update picks smarter chunk sizes for the calculations, so the card wastes less time waiting.

So why care? AMD cards have long lagged behind Nvidia's for running AI locally, which pushed AMD owners toward paid cloud services. Improvements like this mean people with ordinary gaming PCs can run capable models themselves. The practical wins are privacy (your data never leaves your machine), cost (zero subscription), and offline use (it works on a plane). One fun detail: the change was partly written with help from Claude, an AI assistant made by Anthropic — AI helping build the tools that run other AI.

The catch: this is a pre-release nightly build, not a polished product, and it can break. The speed gains apply to specific AMD cards and specific models, not everything. And for most people, big cloud services like ChatGPT or Gemini still deliver better answers with zero setup. Think of this as infrastructure quietly improving, not a product you should rush to install.