NVIDIA Blackwell Gated DeltaNet Gets 1.58x Speedup via AI-Assisted Optimization

A team of researchers from MSInfer has published a technical report detailing their contest-winning optimization of Gated DeltaNet on NVIDIA's Blackwell architecture (B200). In the MLSys 2026 FlashInfer Contest, their AI-assisted approach delivered a 1.58x speedup over baseline, achieving average decode latencies of just 9.315 microseconds and prefill latencies of 239.48 microseconds. The key insight? AI-assisted GPU optimization cannot be reduced to a naive loop of generating faster CUDA kernels and benchmarking. Instead, the team emphasizes that contest-grade performance requires holistic system thinking — including structural reformulation of the algorithm, specialized workload handling, precise measurement tooling, and careful alignment with the evaluator's benchmarking methodology.

The report directly challenges the prevailing narrative that large language models can simply be prompted to produce optimized GPU code in isolation. The MSInfer team shows that local kernel improvements plateau quickly when the underlying workload demands fundamental algorithmic changes. Their workflow incorporated algorithm design, build system tweaks, and even the interpretation of benchmark results through a human-in-the-loop. For professionals working on GPU-accelerated AI, this case study underscores that the biggest gains often come from rethinking the entire optimization pipeline — not just generating more lines of CUDA. As AI-assisted programming tools proliferate, this end-to-end systems perspective will likely become the differentiator between incremental gains and breakthrough performance.