Kernel Forge automates CUDA kernel optimization with up to 2.83x speedups

Machine learning models spend most of their runtime on a small set of compute kernels like matrix multiplication and convolution. Optimizing these kernels traditionally requires expert engineers to write low-level CUDA code. Kernel Forge, introduced by researchers from multiple institutions, is an open-source agent harness that automates this process. It accepts any unmodified PyTorch model and uses Monte Carlo Tree Search (MCTS) to explore multiple optimization paths, rather than a single linear refinement chain. It supports vision, diffusion, and LLM workloads, and ships with a graphical user interface for monitoring progress, inspecting candidate kernels, and debugging failures. The system is evaluated on four PyTorch models running on an NVIDIA DGX Spark with GB10 GPU.

With only 50 optimization iterations per kernel, Kernel Forge optimized 14 kernels to outperform PyTorch eager mode. Specific speedups include 1.52× on adaptive_avgpool2d in ResNet-50, 1.70× on group_norm in Stable Diffusion 3.5 Medium, 2.83× on softmax in Gemma 4 E2B, and 1.54× on softmax in Qwen 3.5 35B-A3B. The code is publicly available, making state-of-the-art kernel optimization accessible to developers without deep CUDA expertise. This represents a significant step toward automated performance tuning for AI workloads, potentially reducing latency and cost for a wide range of production models.