K-Search translates CUDA kernels to Apple's MLX with 20x speedup
K-Search unlocks 20x faster Mamba SSM kernels on Apple Silicon by translating CUDA expertise automatically.
K-Search, an evolutionary kernel optimization framework originally developed at UC Berkeley Sky Lab, has been extended to support Apple's MLX framework, enabling automatic translation of CUDA kernel expertise to Apple Silicon. The new MLX backend leverages a structured CUDA-to-MLX translation layer that adapts existing CUDA kernels (like attention and state space models) into high-performance GPU kernels for Apple's M-series chips, avoiding the need to rebuild optimizations from scratch.
The framework achieves near-expert level performance, with a 0.97x speedup compared to native MLX Attention kernels, and up to a 20x prefill speedup over the community mlx-lm implementation for Mamba SSM kernels. K-Search uses an iterative optimization loop where an LLM (Gemini 3.5 Pro Preview) acts as a GPU kernel performance engineer, proposing and refining optimizations guided by a hardware-specific specification document. This approach is not limited to MLX and can be applied to other ecosystems where CUDA expertise is transferable, addressing a growing challenge as AI hardware diversity accelerates.
- K-Search now supports Apple's MLX framework, translating CUDA kernel expertise to Apple Silicon automatically, achieving up to 20x speedup for Mamba SSM kernels.
- The tool uses an evolutionary optimization loop with an LLM (Gemini 3.5 Pro Preview) to iteratively refine GPU kernels based on hardware-specific constraints.
- Near-expert performance (0.97x speedup over native MLX Attention) demonstrates the effectiveness of the structured CUDA-to-MLX translation layer.
Why It Matters
Bridges the performance gap in local AI inference on Apple Silicon, unlocking faster and more efficient models for millions of M-series devices.