Nvidia's Qwen3.6-27B-NVFP4: 27B model runs 4x faster with FP4 quantization

Nvidia has open-sourced Qwen3.6-27B-NVFP4, a 27-billion-parameter language model quantized using its proprietary FP4 (4-bit floating-point) format. The model is hosted on Hugging Face and is designed for deployment on Nvidia's H100 and Blackwell GPUs, leveraging Hopper's FP4 tensor core support. Compared to the native BF16 version, the FP4 variant reduces memory footprint from roughly 54 GB to under 14 GB per copy, enabling multi-instance serving on a single GPU. Nvidia claims less than 1% degradation in perplexity across key benchmarks, a rare feat for such aggressive quantization.

Practically, this means developers can run a state-of-the-art 27B model on a single H100 with significantly lower latency and cost. The model builds on the Qwen3.6 architecture (a collaboration with Alibaba's Qwen team) and supports 128K token context windows. Early benchmarks show it outperforms similar-sized FP8 quantized models by up to 20% on reasoning tasks while using half the memory. This release signals Nvidia's push to make large models viable for edge and real-time applications without sacrificing quality.