llama.cpp b9938 adds fast math for AMD GPUs, boosting speed
New GitHub release enables unsafe math optimizations for AMD GPUs, avoiding NaN issues.
The open-source llama.cpp project, led by ggml-org, released version b9938 on July 9, 2025. The key change is enabling the '-funsafe-math-optimizations' compiler flag for AMD/HIP GPU builds. Previously, CUDA builds used fast math for performance, while AMD/HIP did not. Directly applying '-ffast-math' caused compilation errors and NaN outputs because llama.cpp uses INFINITY for masking. The new flag gives most of the speedup without those issues, a contribution from Mark Caldwell.
This release also updates builds across multiple platforms: macOS (Apple Silicon and Intel), Linux (CPU on x64/arm64/s390x, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16), Windows (CPU, CUDA 12 & 13, Vulkan, OpenCL Adreno, HIP), Android arm64, and openEuler. The update is signed with GitHub’s verified signature. For AI practitioners running LLMs locally on AMD hardware, this change can translate to noticeably faster inference times, making local deployment more competitive with NVIDIA CUDA systems.
- Enables -funsafe-math-optimizations for AMD/HIP GPUs, matching CUDA's fast math performance
- Avoids NaN issues that occur with -ffast-math by not requiring -ffinite-math-only
- Builds available for Windows, Linux, macOS, Android, and openEuler across CPU and GPU backends
Why It Matters
AMD GPU users get a speed boost for local LLM inference, narrowing the gap with CUDA.