NVIDIA's Nemotron open data: Scaling agents with 10T synthetic tokens

NVIDIA is tackling the core challenge of agentic AI: the real world doesn't behave like a benchmark. An agent that can't recover from a broken API call or an unseen workflow is just an autocompleter with tools. To bridge that gap, the company released Nemotron open data, a massive collection spanning over 10 trillion pre-training tokens and millions of post-training samples across domains like software engineering traces, tool-use failures, multi-step reasoning, and safety. Synthetic data plays a key role – Nemotron-CC and Nemotron-CC-MATH leverage synthetic content to enhance Common Crawl and improve reasoning, respectively. At ICML 2026, nearly 145 papers cited Nemotron models, underscoring the community's demand for open, reproducible data as a foundation for agent behavior.

NVIDIA's approach also tackles the tension between sharing and secrecy. As VP Bryan Catanzaro notes, every company has a secret workflow or corpus that makes its AI useful – but exposing that directly is risky. Synthetic data allows teams to preserve useful signals without leaking proprietary sources. To help developers explore what's actually in the data, NVIDIA built the Nemotron Post-Training v3 Prompt Atlas, an interactive visual map where each point is a prompt sample, color-coded by dataset, domain, or tool use. Semantically similar prompts cluster together, making it easy to zoom in on coding algorithms, safety, math, or agentic behavior. This tool empowers teams to curate data, build evals, and understand why a model behaves the way it does – all critical for creating agents that are inspectable, explainable, and truly robust.