b8100
The popular open-source inference engine now supports modern BERT architectures for improved text representation.
Deep Dive
The ggml-org team behind llama.cpp released version b8100, adding full modern BERT support. This includes implementing GELU activation in rank pooling and mean calculation before the classifier head. The update allows developers to run modern BERT variants (like those from Hugging Face) more efficiently across CPUs and GPUs, expanding the toolkit's capabilities beyond just Llama-family models for better embedding and classification tasks.
Why It Matters
Enables more efficient, local deployment of state-of-the-art embedding models for RAG and semantic search applications.