llama.cpp b9690 adds Metal rope_back operator for faster Apple inference
Optimizes RoPE rotation on Apple GPUs by reusing kernels with AI assistance.
The latest release of llama.cpp, b9690, introduces a critical performance optimization for Apple Silicon devices: the rope_back operator on Metal. Rotary Position Embedding (RoPE) is a core component of modern LLMs, and the ability to efficiently compute backward rotations is essential for certain model architectures. The team reused existing rope kernels and added a function constant to toggle direction, eliminating duplicate kernel code. This approach reduces GPU memory overhead and speeds up inference on macOS and iOS devices.
The release also highlights a collaborative milestone: it was assisted by the Qwen3.6-27B model itself. This integration demonstrates how LLMs can actively contribute to optimizing the very infrastructure running them. The commit is signed and verified, and the build matrix spans macOS (Apple Silicon and Intel), Linux (x64, arm64, s390x, with Vulkan, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), Android, and iOS XCFramework. Ongoing development continues to refine rope operations and expand hardware support.
- Implements rope_back operator for Metal, enabling efficient backward RoPE rotation on Apple GPUs.
- Reuses existing rope kernels with a function constant toggle, reducing code duplication and improving maintainability.
- Assisted by Qwen3.6-27B model, showcasing AI-driven development in open-source LLM infrastructure.
Why It Matters
Apple Silicon users gain faster local LLM inference, making llama.cpp even more viable for edge AI applications.