llama.cpp v9885 delivers 2x speed boost on AIX systems
New tiled matmul fix doubles prompt processing speed on IBM's Unix.
The ggml-org team released llama.cpp b9885, addressing a critical crash in tiled matrix multiplication on AIX (IBM's Unix OS). The matmul_tiled path uses large local stack buffers for A_pack and B_pack, which triggered segmentation faults on AIX. By reducing the buffer footprint, developers can now run large language models on AIX without crashes, unlocking a ~2x speed improvement in prompt processing (PP_Speed) for FP32, Q4_0, and Q8_0 quantizations.
Testing with Llama 3.2 3B (FP32) and Qwen 2.5 3B (Q4_0 and Q8_0) showed consistent gains across llama-bench, llama-batched-bench, and llama-cli. The release also updates platform support for macOS Apple Silicon (including KleidiAI), Linux x64/arm64/s390x with Vulkan, ROCm, OpenVINO, SYCL, Android arm64, Windows x64/arm64 with CUDA 12/13, Vulkan, OpenCL, and HIP. AIX users finally get the same tiled matmul benefits available on other architectures.
- Fixes segmentation fault on AIX by reducing stack buffer size for matmul_tiled
- Delivers ~2x faster prompt processing on FP32, Q4_0, and Q8_0 models
- Tested with Llama 3.2 3B and Qwen 2.5 3B across multiple benchmarks
Why It Matters
Enterprise AIX users can now run local LLMs 2x faster, unlocking better inference on legacy Unix infrastructure.