LocalAI 4.5.0 adds depth perception, sound tagging, on-device TTS, and more

LocalAI 4.5.0 expands the platform's sensory capabilities with four new backends. The depth-anything backend (a native Go gRPC wrapper around a ggml port of Depth Anything 3) provides monocular metric depth estimation and camera pose from a single CPU, exposed via a typed Depth RPC and a new POST /v1/depth endpoint that returns depth maps, statistics, and 3×4 extrinsic/intrinsic matrices. The CED backend tags 527 AudioSet sound classes (baby cry, glass breaking, alarms) over POST /v1/audio/classification and supports VAD-decoupled realtime streaming. For speech, the supertonic ONNX TTS backend delivers multilingual output with 10 voices, fast CPU cold start, and no espeak/G2P dependency. Privacy gets a boost with privacy-filter.cpp, a new NER engine that scans conversations for named entities and secrets using restricted-regex detection, with a UI-based PII editor.

Multi-user serving is now zero-config: prefix caching runs by default, Blackwell-optimized batch size (2048), and VRAM-scaled n_parallel enable continuous batching out of the box without KV cache blow-up. The redesigned UI ships end-to-end with a calmer interface, new console with built-in assistant, operations dashboard, sortable mobile tables, and unsaved-changes guards. Model aliases allow live redirection of model names without client reconfiguration. Word-level ASR timestamps are now available for NeMo and CrispASR backends, plumbed through gRPC. Distributed staging is hardened: cold-load staging detaches from request context, progress broadcasts across replicas, and the downloader is now resumable. Additionally, 8 Depth Anything 3 GGUFs (base/small/large/giant) are available at mudler/depth-anything.cpp-gguf.