Llama.cpp Update Gives You More Control Over AI Image Processing

Most people use AI through the cloud, like ChatGPT or Google. But a growing number of tools let you run AI directly on your own laptop or phone. Llama.cpp is one of the most popular free tools for this. It powers apps that can chat, write, and even understand images — all without sending your data to a server. This week, the team behind llama.cpp released a new update that adds a handy feature: the ability to pick exactly which part of your computer processes image-related tasks.

In plain terms, when an AI looks at a photo or a document, it uses two separate pieces: the main brain that reads text and a special helper that interprets images. That helper is called a 'multimodal projector' (or 'mmproj' for short). Before this update, the helper always ran on the same device as the main AI, which could slow things down or waste memory. Now, you can tell it to run on your graphics card instead of your main processor, or vice versa. That means you can optimize for speed, battery life, or memory based on what you're doing.

The update also includes a backup option — an environment variable (a system setting) called MTMD_BACKEND_DEVICE — that does the same thing for people who prefer a more permanent configuration. And it's all backwards compatible, so older projects won't break. The fine print: this is a tool for developers, so you won't see a shiny button. But if you use an app built on llama.cpp, this update means smoother performance and better hardware use. For everyone else, it's another step toward making offline AI faster, cheaper, and more private on the devices you already own.