NVIDIA's Nemotron 3.5 Lightning hits SageMaker with 4x agent throughput
NVIDIA's open 30B MoE model delivers 4x throughput using only 3B active parameters.
NVIDIA has launched Nemotron 3.5 Lightning on Amazon SageMaker JumpStart, giving developers one-click access to an open model built for high-volume agentic workloads. The model uses a hybrid Mixture-of-Experts (MoE) architecture with 30B total parameters but only 3B active per forward pass, which lets it run on a single supported GPU while delivering up to 4x higher throughput and 30% faster task completion compared to peers. It also features a 1M-token context window and DFlash speculative decoding to reduce per-token latency, making it suitable for always-on agents that gather context, reason, and act continuously.
Distilled from NVIDIA's frontier Nemotron 3 Ultra, the model is trained specifically for agentic tool use across popular agent harnesses and released on open datasets, so enterprises can customize it with NVIDIA NeMo and own the resulting weights. On benchmarks, it scores 81.94 on MMLU Pro, 75.44 on GPQA Diamond, and 51.56 on SWE-bench Verified, with NVFP4 quantization staying close to BF16 accuracy. Because it handles specialized, high-frequency steps like alert classification or field extraction, it fits into a system-of-models approach, letting organizations route simple tasks to Lightning while reserving frontier models for complex planning and orchestration.
- NVIDIA Nemotron 3.5 Lightning is a 30B-parameter MoE model with only 3B active parameters, deployable on a single GPU via SageMaker JumpStart
- Delivers up to 4x higher throughput and 30% faster task completion on high-volume agentic workloads, with a 1M-token context window
- Open model distilled from Nemotron 3 Ultra, customizable with NVIDIA NeMo, and scores 81.94 on MMLU Pro and 51.56 on SWE-bench Verified
Why It Matters
Enterprises can now run high-volume agent steps cheaply on one GPU, cutting latency and infrastructure costs without frontier-scale compute.