llama.cpp b9760 adds video input and generalized schema
Run video inputs locally with new input_video and base64 support
llama.cpp's newest release, tagged b9760, marks a significant step toward multimodal local AI inference. The core change is a refactored input file schema that generalizes how the library accepts data. Previously limited to text and images, the server now supports an 'input_video' field, accepting raw base64 encoded video. This leverages the same modular infrastructure, making it easier to add future modalities. The commit also includes nits, fixes for Windows, and updates across all major platforms.
The implications for developers are substantial. By moving from ad-hoc input handling to a generalized schema, llama.cpp can now process video frames directly—enabling applications like real-time video analysis, multimedia summarization, and vision-language model interactions entirely on-device. The update supports various backends: CPU on Linux/Windows/macOS, GPU via CUDA 12/13, Vulkan, ROCm 7.2, OpenVINO, SYCL, HIP, and Android arm64. This ensures broad hardware compatibility while maintaining the performance llama.cpp is known for.
- Generalized input file schema replaces hardcoded input types with flexible 'input_video' field
- Accepts raw base64 encoded video data alongside existing text/image inputs
- Available on all major platforms including macOS (Apple Silicon & Intel), Windows, Linux, Android, and multiple GPU backends
Why It Matters
Enables local multimodal AI with video input, reducing cloud dependency for real-time analysis and multimedia tasks.