DeepSeek's ds4 flash writes custom Metal kernel for Kimi K2 in 50 min
An AI model hand-coded a low-level GPU kernel to run a 90GB Kimi K2 quant on a Mac Studio.
A Reddit user, self-described as a "programming ignoramus," shared a striking demonstration of AI autonomy: DeepSeek's ds4 flash (0731) automatically wrote a custom Metal kernel for Kimi K2, quantized to unsloth's IQ1_0 format, in about 50 minutes. The model resorted to this after failing to locate an existing kernel on GitHub for the Mac Studio 512GB. The resulting performance—roughly 4 tokens/s decode and 20 prefill—is "meh" compared to native implementations, but a massive improvement over CPU-only execution, proving the model can bridge a hardware-software gap without human intervention.
Running the same model on MLX at 8 and 4 bits, the user found the 2-bit unsloth quant to be within a common sampling-level margin of error for his tasks. On an RTX Pro 6000 via LM Studio, it delivered about 1k prefill and 50 decode. While the 4-bit GLM 5.2 still edged out Kimi K2 in quality, the gap was narrow. For a 90GB low-bit quant, this is remarkable: an LLM essentially acted as a compiler engineer, generating production-ready GPU code for a model it is itself powering. It raises the bar for what we expect from AI coding agents—not just writing app code, but debugging and optimizing at the hardware abstraction layer.
- ds4 flash (0731) authored a custom Metal kernel for Kimi K2 IQ1_0 in ~50 minutes, with no existing GitHub solution.
- Achieved 4 t/s decode and 20 prefill on Mac Studio 512GB; 1k prefill and 50 decode on an RTX Pro 6000.
- 2-bit unsloth quant quality matched 8/4-bit MLX within sampling margin; 4-bit GLM 5.2 remained slightly ahead.
Why It Matters
AI that writes its own low-level kernels could make massive quantized models runnable on consumer hardware without human GPU engineering.