Scaling seismic foundation models on AWS: Distributed training with Amazon SageMaker HyperPod and expanding context windows
A new distributed training solution achieved near-linear scaling and massively expanded context windows for 3D geological analysis.
Geoscience data leader TGS, in collaboration with the AWS Generative AI Innovation Center, has dramatically accelerated the training of its Vision Transformer-based Seismic Foundation Models (SFMs). The partnership focused on overcoming key bottlenecks: the massive scale of proprietary 3D seismic data, computationally intensive training cycles, and the need to analyze larger geological contexts. By implementing a solution built on Amazon SageMaker HyperPod, the team established a resilient, scalable training infrastructure that streams terabytes of data directly from Amazon S3, eliminating intermediate storage layers and maintaining high GPU throughput.
The core of the performance leap came from optimizing distributed training across a cluster of 16 Amazon EC2 P5 instances, each equipped with 8 NVIDIA H200 GPUs. Using advanced parallelization and context parallelism techniques, the system achieved near-linear scaling, allowing the SFM to process significantly larger 3D volumes than before. This architectural overhaul reduced the model's training time from an estimated six months down to a mere five days. The expanded context window now enables the AI to capture both fine local details and broader geological patterns simultaneously, providing more comprehensive insights for energy exploration and production workflows.
- Training time for TGS's Vision Transformer seismic model was reduced from 6 months to 5 days using distributed training on Amazon SageMaker HyperPod.
- The solution uses a cluster of 16 EC2 P5 instances with 128 NVIDIA H200 GPUs and achieves near-linear scaling for efficient model training.
- Expanded context windows allow the AI to analyze larger 3D geological volumes, capturing both local details and broader patterns for better exploration insights.
Why It Matters
This breakthrough enables faster iteration on complex geological AI models, accelerating discovery and de-risking for the global energy sector.