Event Tensor: A Unified Abstraction for Compiling Dynamic Megakernel

A research team from Carnegie Mellon University and Meta has introduced Event Tensor, a novel compiler abstraction designed to optimize GPU performance for modern workloads like large language model (LLM) inference. The core problem they address is the significant kernel launch overhead and coarse synchronization in GPUs, which limits inter-kernel parallelism. Traditional megakernel techniques fuse multiple operators into a single persistent kernel to eliminate launch gaps, but they struggle with dynamic shapes and data-dependent computations common in real-world AI applications. Event Tensor provides a unified representation that encodes dependencies between computational tasks, enabling first-class support for both shape and data dynamism.

Built on this abstraction, the Event Tensor Compiler (ETC) applies sophisticated static and dynamic scheduling transformations to generate high-performance persistent kernels. In evaluations, ETC demonstrated substantial improvements, achieving state-of-the-art LLM serving latency while significantly reducing system warmup overhead. The compiler's ability to handle dynamic computation patterns makes it particularly valuable for production AI systems where input sizes vary and computations are data-dependent. This research, accepted at MLSys 2026, represents a significant advancement in compiler technology for heterogeneous computing systems.

The technical approach involves representing computations as a graph of tiled tasks with explicit dependency tracking through event synchronization primitives. This allows the compiler to optimize both regular and irregular computation patterns efficiently. The system's performance gains come from reducing GPU idle time between kernel launches and enabling more fine-grained parallelism. For LLM inference specifically, this translates to faster response times and better resource utilization, addressing one of the key bottlenecks in deploying large models at scale.