Helion's PyTorch-native kernels boost vLLM FP8 inference on H100/B200
Helion DSL fuses and optimizes vLLM kernels for major throughput gains on Blackwell.
vLLM, a leading LLM inference framework, relies on custom GPU kernels and fusion passes for high throughput. Helion is a PyTorch-native kernel DSL that uses a tile-programming model, offering a familiar syntax while exposing low-level control over memory layout and scheduling. The team integrated Helion kernels into vLLM for FP8 quantized inference with Qwen3 models, replacing nearly all forward-pass kernels including rms_norm, quantization, and scaled_mm operations. They implemented fused kernels like rms_norm_dynamic_per_token_quant and silu_and_mul_per_block_quant, leveraging Helion's ahead-of-time autotuning to explore optimal configurations for NVIDIA H100 and B200 GPUs.
End-to-end benchmarks revealed throughput gains across multiple serving scenarios compared to vLLM's existing implementations. Dynamic per-token and per-group quantization patterns were supported, with Helion enabling fusions that TorchInductor could not perform across opaque custom operators. The scaled_mm and scaled_mm_blockwise kernels matched or exceeded Triton baselines. Additional optimization work targets GEMM performance on Blackwell GPUs using DeepGEMM integration. Helion's approach reduces the complexity of writing high-performance kernels, making advanced LLM serving optimizations more accessible to the community.
- Helion kernels replaced forward-pass kernels for FP8 quantized inference in vLLM with Qwen3 models.
- Throughput gains demonstrated on NVIDIA H100 and B200 GPUs via fused kernels like rms_norm+quant and silu_and_mul+quant.
- Helion's AOT autotuning and PyTorch-native tile programming enable productive high-performance kernel development.
Why It Matters
Simplifies building high-performance LLM serving kernels, accelerating deployment of efficient AI inference on latest GPUs.