llama.cpp b10037 adds manual tensor types in quant with --pure flag
New release lets you fine-tune quantization by specifying tensor types manually.
llama.cpp, the popular open-source C++ implementation for running LLaMA and other transformer models locally, has released version b10037. The key update is a new quantization feature: `--pure` now allows manual tensor types. Previously, quantization was fully automatic; now users can specify exact data types for individual tensors (e.g., Q4_0, Q8_0, f16) when using the pure quantization mode. This is critical for performance tuning on specific hardware, especially when mixing precision for different layers to balance speed and quality.
The release delivers builds across virtually every major platform. Apple users get binaries for Apple Silicon (arm64, with KleidiAI option disabled by default), Intel x64, and iOS XCFramework. Linux supports x64/arm64 CPU, s390x, plus GPU backends Vulkan, ROCm 7.2, OpenVINO, and SYCL (FP32/FP16). Windows adds CUDA 12 and 13 (separate DLLs), Vulkan, OpenCL Adreno for ARM, SYCL, and HIP. Android users get arm64 CPU builds. openEuler builds (x86/aarch64) target Ascend 310P and 910B with ACL Graph. The release is signed with GitHub verified signature (B5690EEEBB952194).
- Manual tensor types now allowed with --pure flag for fine-grained quantization control
- Supports macOS, Linux, Windows, Android, and openEuler across CPU and GPU backends
- Pre-built binaries for Apple Silicon, CUDA 12/13, ROCm, Vulkan, OpenVINO, SYCL, and HIP
Why It Matters
Gives developers precise control over LLM quantization, improving efficiency on diverse hardware.