llama.cpp Update Makes Local AI Faster on Intel Machines

There's a popular free program called llama.cpp that lets you run AI models directly on your own computer — no internet, no cloud, no monthly fee. It's what many people use to chat with AI privately on their PC or laptop. The latest update, version b10669, makes that experience noticeably smoother on machines with Intel graphics.

The update targets a behind-the-scenes step called the KV cache, which is like AI's short-term memory. During a conversation, the AI has to repeatedly read and update this memory. Previously, on some Intel chips, the software was copying the entire memory back and forth every time — imagine rewriting all your notes before every sentence. The new version keeps the memory in place and works with it directly. The developer measured the improvement: memory traffic dropped from a huge 4.56 gigabytes down to under a quarter of a gigabyte for the first chunk of text.

Why should you care? Because copying less data means the AI responds faster and uses less memory. If you've ever run an AI on your laptop and watched it slow down or hog all your RAM, this is the kind of fix that helps. It also means more people can run AI on older or mid-range Intel machines without buying expensive hardware.

The catch: this optimization only applies to a specific technical path (oneDNN flash attention on Intel SYCL). If you're using an Nvidia or AMD card, or just using AI through a website, this update doesn't change anything for you. Still, it's a sign that the tools for running AI at home keep getting cheaper and more efficient.