llama.cpp v9777 adds LFM2.5 embedding models for local AI
Two 350M-parameter embedding models now run locally on CPU/GPU
Deep Dive
llama.cpp release b9777 adds LFM2.5-ColBERT-350M and LFM2.5-Embedding-350M models, restores LFM2 models in README.md, and provides platform builds for macOS (Apple Silicon, Intel, iOS XCFramework), Linux (Ubuntu x64/arm64/s390x, Vulkan, ROCm, OpenVINO, SYCL), Android arm64, Windows (x64/arm64, OpenCL, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), and openEuler (disabled). UI assets included. One user reacted with a hooray emoji.
Key Points
- LFM2.5-ColBERT-350M enables efficient dense retrieval via late interaction, ideal for RAG pipelines.
- LFM2.5-Embedding-350M provides general-purpose semantic embeddings with 350M parameters.
- Available across Windows, macOS, Linux, Android, and iOS with CPU, CUDA, Vulkan, and Apple Silicon support.
Why It Matters
Privacy-focused local embeddings for RAG and search, no cloud required.