LocalAI v4.4.0 adds audio backends, video understanding, and distributed routing

LocalAI v4.4.0 is a landmark release that pushes the open-source AI platform into new territory. The headline additions are two brand-new audio backends: parakeet-cpp wraps NVIDIA's NeMo Parakeet (FastConformer TDT/CTC/RNNT) for high-quality CPU-based ASR with streaming and word timestamps, while crispasr is a multi-architecture backend that handles both ASR and TTS from a single binary, supporting Whisper, Parakeet, Moonshine, and more. On the vision side, video understanding lands in llama-cpp via mtmd, and video generation is now possible through stablediffusion-ggml with LTX-2. Object detection and segmentation get a native backend (rfdetr-cpp) with 32 prebuilt GGUFs and per-detection PNG masks.

Distributed mode gets significant upgrades: prefix-cache-aware routing is enabled by default, file transfers are now resumable, and NATS JWT authentication with TLS/mTLS is supported. The new intelligent middleware layer enables capability-based model routing, PII detection and redaction, and cloud-model proxying (including a MITM proxy for Claude Code/Codex subscription auth). Security hardening closes a credential-leak class (CVE-like) across every outbound HTTP client. Other highlights include an interactive local-ai chat CLI, RAG source citations for agents, TTS per-request control instructions, and support for new models like Gemma 4 QAT bundles, Ideogram4, and LTX-2.3 22B GGUFs.