LocalAI 4.8.0 drops 3 new modalities, vLLM.cpp engine
LocalAI 4.8.0 ships vLLM.cpp, 3D generation, audio.cpp, and 22-day dev sprint
LocalAI, the open-source alternative to OpenAI's API ecosystem, just shipped version 4.8.0 with a massive 22-day development sprint that merged 386 pull requests. The headline feature is vLLM.cpp, a C++20-native engine that began as a vLLM port but now operates independently with identical output guarantees. This alpha-stage backend delivers 1.045x vLLM performance on Qwen3-27B at concurrency 1, with zero Python or PyTorch dependencies. Apple Silicon users get 1.5x to 2.2x speedups via MLX acceleration, and the engine supports CPU, CUDA, Metal, and Vulkan backends.
The update also introduces three new modalities: 3D generation via trellis2cpp, a unified audio.cpp backend serving six audio tasks across model families, and sub-2-bit model support via the new bonsai backend. LocalAI 4.8.0 adds 7 new backends, 294 new gallery entries, VRAM caps (80% or 12GB), and two new TTS engines (magpie-tts-cpp and moss-tts-cpp). The web UI shrinks by 3.48x, trace endpoints slim from 21MB to 7KB, and distributed mode gets reliability hardening. The team also launched a new localai.io portal and overhauled documentation for easier onboarding.
- vLLM.cpp (alpha) delivers 1.045x vLLM performance with no Python/PyTorch, supporting CPU/CUDA/Metal/Vulkan and MLX acceleration
- New modalities: 3D generation via trellis2cpp, audio.cpp backend for 6 audio tasks, and sub-2-bit model support via bonsai
- VRAM caps (80% or 12GB), 7 new backends, 294 gallery entries, and 3.48x smaller web UI with 7KB trace endpoints
Why It Matters
LocalAI 4.8.0 brings production-grade multi-modality to local LLM deployments with performance parity to vLLM and 30% smaller footprint.