Exploring Novel Data Storage Approaches for Large-Scale Numerical Weather Prediction
A new object storage system processes massive weather data 10x faster than traditional file systems.
Researcher Nicolau Manubens Gil's PhD thesis for ECMWF tested novel object storage systems for large-scale Numerical Weather Prediction (NWP) and AI workloads. The study benchmarked DAOS and Ceph against the standard Lustre file system. DAOS demonstrated superior scalability and flexibility for high-performance I/O, enabling faster processing of ever-increasing weather data volumes. This provides a new storage architecture option for HPC centers running demanding AI and simulation applications.
Why It Matters
Faster data handling accelerates critical weather forecasting and climate modeling, improving prediction accuracy for industries and disaster response.