ARGUS: Agentic GPU Optimization Guided by Data-Flow Invariants
New AI system writes GPU kernels achieving 99-104% of expert-level performance, 2-1543x faster than other agents.
A research team from Stanford and other institutions has introduced Argus, a novel AI framework that solves a critical bottleneck in AI-assisted programming: generating high-performance GPU code. While LLM-based coding agents can produce functionally correct kernels, their performance typically lags far behind hand-optimized libraries for compute-intensive operations like matrix multiplication (GEMM), attention mechanisms, and Mixture-of-Experts (MoE) layers. Argus bridges this gap by employing "data-flow invariants"—compile-time specifications that encode how data must move through the GPU's memory hierarchy and execution units. This allows the system to reason about complex, interdependent optimizations like tiling, shared-memory staging, and software pipelining, which are essential for peak performance.
Argus provides a tile-based, Pythonic Domain-Specific Language (DSL) that exposes hardware instructions while abstracting low-level complexity. When the AI agent violates a performance-critical constraint, the compiler returns a concrete counterexample, pinpointing the exact thread, data element, and program point of failure. This dense, structured feedback enables targeted fixes, a significant improvement over the sparse pass/fail signals used by other systems. The invariants are verified at compile time using abstract interpretation and SMT solving, adding zero runtime overhead.
In evaluations on the AMD MI300X GPU, Argus-generated kernels for GEMM, flash attention, and MoE operations achieved 99% to 104% of the throughput of state-of-the-art, hand-written assembly code. These operations account for over 90% of GPU time during LLM inference. The framework demonstrated massive speedups of 2x to 1543x compared to existing agentic systems like GPT-Engineer or SWE-agent. Furthermore, Argus showed strong generalization, solving 100% of Level 1 and 90% of Level 2 tasks in the 200-task KernelBench suite, proving its ability to handle a wide range of GPU programming challenges.
- Achieves 99-104% of hand-optimized assembly performance for critical LLM kernels like GEMM and flash attention on AMD MI300X GPUs.
- Uses data-flow invariants and a Pythonic DSL to provide structured compiler feedback, enabling 2-1543x speedups over other AI coding agents.
- Solves 100% of Level 1 and 90% of Level 2 tasks in the KernelBench suite, demonstrating strong generalization across GPU programming problems.
Why It Matters
This could dramatically accelerate AI development by automating the creation of high-performance, hardware-specific code, reducing reliance on scarce expert engineers.