Developer Tools

Why llama.cpp's Latest Update (b9711) Finally Makes LLaVA-UHD Image Handling Consistent

LLaVA-UHD now always uses overview image first in llama.cpp's multimodal pipeline.

Deep Dive

The ggml-org/llama.cpp project has released version b9711, focusing on refining multimodal image processing for LLaVA-UHD (Large Language and Vision Assistant for Ultra-High Definition images). The primary change is in the mtmd module's image preprocessing: it now forces the 'overview' image to be processed first (ov_img_first) during the splitting of high-resolution inputs. This refactor resolves inconsistencies in how overview patches are ordered when feeding into the vision encoder, leading to more stable outputs for models like LLaVA-1.6 and LLaVA-UHD.

Alongside the algorithmic fix, the release ships precompiled binaries across an extensive range of platforms: macOS Apple Silicon (with and without KleidiAI acceleration), macOS Intel, iOS XCFramework, Linux x64/arm64/s390x CPU builds, plus GPU-accelerated variants (Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16), Android arm64, and Windows x64/arm64 with CUDA 12/13, Vulkan, HIP, and OpenVINO. The release also includes UI assets and an openEuler build pipeline with ACL Graph support. This broad support makes the LLaVA-UHD optimization immediately usable for developers running vision-language models locally on diverse hardware.

Key Points
  • Refactored mtmd preprocessing to always apply ov_img_first for LLaVA-UHD, fixing overview patch ordering
  • Prebuilt binaries for macOS, Linux (x64/arm64/s390x), Windows (CPU/CUDA/Vulkan), Android, and iOS
  • GPU acceleration support includes ROCm 7.2, Vulkan, OpenVINO, SYCL, HIP, and CUDA 12.4/13.3

Why It Matters

Developers running LLaVA-UHD locally now get more consistent image processing, critical for reliable multimodal inference on consumer hardware.

📬 Get the top 10 AI stories daily