Developer Tools

llama.cpp b9573 fixes Plamo2 regression, ships multi-platform builds

The open-source LLM runner gets a key Plamo2 attention fix and broad platform support.

Deep Dive

The llama.cpp open-source project released version b9573, a maintenance release that addresses a specific regression in Plamo2 model support. The fix resolves an issue with attention_key/value_length calculations that could cause incorrect inference in Plamo2-based models. This is a critical patch for users running the Plamo2 architecture locally, as it ensures the attention mechanism functions correctly across long context windows.

The release also provides pre-built binaries for a wide range of platforms, including macOS (Apple Silicon and Intel), iOS, Linux (x64 and ARM64), Windows (x64 and ARM64), Android, and openEuler. Backend support spans CPU, Vulkan, CUDA, ROCm, OpenVINO, SYCL, and HIP, making it easy to deploy on GPUs and accelerators. The build matrix shows that some configurations (e.g., KleidiAI, SYCL FP32, HIP) are currently disabled, but the majority are active. This release continues llama.cpp's mission of enabling high-performance local inference for popular LLM architectures.

Key Points
  • Fixes a Plamo2 attention_key/value_length regression that could degrade model output quality
  • Pre-built binaries for macOS, iOS, Linux, Windows, Android, and openEuler across CPU and GPU backends
  • Active support for Vulkan, CUDA 12/13, ROCm 7.2, OpenVINO, and SYCL for hardware acceleration

Why It Matters

Ensures stable local inference for Plamo2 models and expands hardware compatibility for DIY LLM deployments.

📬 Get the top 10 AI stories daily