Developer Tools

llama.cpp release b9562 adds video input support for multimodal AI

Open-source LLM runner now accepts video alongside text for local inference.

Deep Dive

llama.cpp, the widely-used open-source project for running large language models locally (116k stars, 19.4k forks), has released version b9562 with a major new capability: video input support. The update, tagged b9562 and signed with GitHub's verified signature, introduces the mtmd (multimodal) module that accepts video as input. Users can now feed video data via base64 encoding when using the server endpoint, or pass video files directly through the new --video argument in the command-line interface. This enables local inference on multimodal LLMs that can process visual temporal information, not just static images or text.

The release includes builds for macOS (Apple Silicon and Intel), Linux (x64, arm64, s390x with Vulkan, ROCm, OpenVINO, SYCL support), Windows (x64 and arm64 with CUDA 12/13, Vulkan, HIP), Android arm64, and iOS XCFramework. The commit also adds a MTMD_VIDEO configuration flag, updates documentation, and includes a lazy bitmap API. This feature is particularly significant as it allows developers to run video understanding models entirely offline and on consumer hardware, expanding the use cases for local AI assistants, media analysis, and edge computing without relying on cloud APIs.

Key Points
  • Video input support added via mtmd module, accepting base64 on server or --video CLI flag
  • New MTMD_VIDEO config and updated documentation for multimodal inference
  • Builds available for macOS, Linux, Windows, Android, and iOS with GPU acceleration options

Why It Matters

Enables local multimodal AI inference with video, expanding offline use cases for developers.

📬 Get the top 10 AI stories daily