llama.cpp b9855 boosts CPU inference with AVX2 nvfp4 optimization
New dot product routine uses UE4M3 LUT for faster 4-bit quantized models.
Deep Dive
llama.cpp b9855 released with AVX2 optimization for nvfp4 dot product using UE4M3 LUT.
Key Points
- Adds AVX2-optimized dot product for nvfp4 4-bit quantized models
- Uses UE4M3 lookup table to reduce memory bandwidth and compute cycles
- Available on all major platforms: Windows, Linux, macOS, Android (x64/ARM64)
Why It Matters
Faster local LLM inference on consumer CPUs reduces GPU dependency and lowers barrier for private AI deployment.