New Version of llama.cpp Makes AI Run Smoother on More GPUs
If you run AI on your own computer, this update fixes a frustrating bug.
Most people use AI through websites like ChatGPT, but there's a growing movement to run AI directly on your own computer. llama.cpp is one of the most popular free tools for doing exactly that. It lets you download a model and use it privately, without sending your data to a company. This week, the developers released version b10639, a pre-release that fixes an important bug for people with certain graphics cards.
The fix is technical, but here's the simple version. Graphics cards process information in small groups called "warps." Most cards handle up to 64 items at a time, but some newer cards handle more. The old version of llama.cpp didn't know what to do with those bigger groups, causing crashes or slowdowns. The new version simply clamps the group size to 64, which avoids the problem and keeps things stable across a wider range of hardware.
The update also includes ready-made downloads for many systems, including Windows, Mac, Linux, and Android, with support for different hardware setups. Since this is a pre-release, it's not the stable version yet, so average users might want to wait. But for hobbyists and developers who run AI locally, this is great news: it means AI will work more reliably on more devices, giving people more control and privacy over their AI tools.
- llama.cpp lets you run AI models on your own computer instead of sending data to the cloud.
- The b10639 update fixes a bug with graphics cards that use larger processing groups, preventing crashes.
- It's a pre-release version with downloads for Windows, Mac, Linux, and Android, so it's meant for testing.
Why It Matters
More reliable local AI means better privacy and control over how you use artificial intelligence.