Non-invasive malleability reduces node-hours on TOP500 supercomputers

Researchers from the Barcelona Supercomputing Center and the University of Valencia introduce the Dynamic Management of Resources (DMR) framework, a practical approach to MPI malleability for production high-performance computing (HPC) clusters. Current supercomputers allocate rigid, fixed-size resources for scientific applications even though many exhibit time-varying behavior. DMR sidesteps the need for scheduler modifications or deep code rewrites by integrating with vanilla resource managers and existing MPI libraries. The methodology achieves non-invasive malleability: applications can dynamically adjust their resource footprint during runtime without requiring application developers to alter the original code. This lowers the barrier to adopting dynamic resource management in real-world HPC environments.

The team tested DMR on three TOP500 supercomputers—including MareNostrum 5 and two others—under realistic production configurations using two large-scale scientific applications. Results show that DMR matches the performance of static (fixed-size) allocations in controlled environments while substantially reducing total node-hour consumption for the same workload. By enabling fine-grained resource elasticity, the framework allows HPC centers to improve utilization and energy efficiency without sacrificing throughput or requiring expensive scheduler upgrades. The paper demonstrates that malleability is not just a laboratory concept but a viable, deployable solution for modern supercomputing centers seeking to do more with less.