New llama.cpp Update Makes AI Run Faster on AMD GPUs

llama.cpp is a hugely popular free program that lets you run AI models like ChatGPT-style assistants directly on your own laptop or desktop—no internet needed. This keeps your data private and avoids monthly fees. The project just pushed out a pre-release version (b10720) that includes a subtle but meaningful speed boost for AMD graphics cards.

Here's the technical bit in plain language: AI models often need to pick the top few most likely answers from a huge list. The software uses a sorting method called TOP_K to do this. The new update uses a faster sorting technique, called radix sort, specifically when the list is very long. On AMD GPUs, which communicate through a system called ROCm, this can shave time off every response.

The practical effect? If you're running a local chatbot, an image generator, or any AI tool on an AMD graphics card, you might notice responses coming back a bit quicker than before. For developers, it's a small optimization in a long-running project. For regular users, this means local AI becomes slightly more practical—faster interactions make it feel more natural and less like waiting for a web page to load.

It's still a pre-release, so not every AMD setup will see dramatic gains. But it shows how the open-source community keeps polishing the experience of running AI at home. Over time, these small updates add up to a real difference in speed and usability.