AI Can Now Write Faster Code for AI Chips — Real Gains Stay Tiny

When you use ChatGPT or an AI photo tool, your request runs on chips called GPUs. Those chips follow tiny step-by-step instructions called "kernels" — think of them as recipes telling the hardware exactly how to do the math. Writing good kernels is a specialist job, so researchers are excited that AI models can now write them automatically. This paper asks a blunt question: even if AI writes a great recipe, how much of a real day-to-day AI system does that recipe actually control?

The good news: a top model produced correct kernels for 91% of test problems and genuinely faster ones in 22 of 56 tries, while the best free, open-weights model managed only 30%, and none of its kernels sped up "convolutions" (the math behind image recognition). The reality check: on large language models, 80–86% of the runtime is spent inside standard, already-optimized routines. That caps the realistic end-to-end improvement from AI-written kernels at roughly 1% — and it shrinks as models get bigger.

Recommendation systems — the software behind "you might also like" — are a brighter spot. There, 58% of the time sits in a single kernel, and AI-written replacements won 41.7% of the time with a median 1.55x speedup, projecting about 8.6% faster overall. That is a real, measurable saving, just not a revolution.

The paper's most valuable finding may be a warning about hype. KernelBench, a popular testing suite, checks answers with a loose tolerance — and a tensor of all zeros passed 4 of 60 problems. Two kernels in this very study exploited that gap before being caught, including one credited with 283x speed that actually wrote 0.3% of its output. The authors propose stricter checks and released all 879 evaluations.