Amazon SageMaker AI adds NVIDIA Blackwell GPUs for faster 1B-64B model training

Amazon SageMaker AI has added support for P6-B200 instances powered by NVIDIA Blackwell GPUs, giving AI teams a significant upgrade for training large models. Blackwell’s dual-chip architecture and fifth-generation Tensor Cores, combined with NVLink 5 interconnects delivering 1.8 TB/s bidirectional bandwidth, allow larger batch sizes and longer sequence lengths without aggressive model sharding. With 180 GB of HBM on B200 and 268 GB on B300, memory pressure is substantially reduced. The post details how to configure training jobs for transformers from 1B to 64B parameters using PyTorch FSDP, covering batch size tuning, precision format selection, and activation checkpointing. By reducing communication overhead and enabling single-node training for models that previously required multi-node setups, Blackwell on SageMaker AI can accelerate iteration cycles and cut infrastructure costs.

For practitioners, the key levers are memory management: larger batch sizes improve throughput by reducing gradient synchronization steps; simplified sharding minimizes inter-GPU communication; and longer sequence lengths benefit tasks requiring long-range context. Activation checkpointing (with 10-30% compute overhead) frees memory for bigger batches or sequences. The guide recommends starting with batch size if throughput is the priority, simplifying sharding if communication is the bottleneck, or prioritizing sequence length for context-dependent tasks. With Flexible Training Plans, users get predictable access and cost management. This integration makes it practical to train large models with fewer nodes, directly addressing common constraints in distributed training.