New UM-Bridge load balancer cuts idle time to near zero for Bayesian inversion
Dynamic load balancing achieves average node idle time of close to a millisecond.
A team of researchers (Loi, Wille, Reinarz) has developed an improved dynamic load balancer for the UM-Bridge framework, designed to address the scheduling challenges of uncertainty quantification (UQ) workflows on high-performance computing (HPC) systems. UQ tasks generate many heterogeneous model evaluations with loose but non-trivial dependencies, which traditional static schedulers handle poorly. The new load balancer dynamically distributes sampling requests without prior assumptions about workload characteristics, achieving an average node idle time of close to one millisecond.
The method was validated on a realistic Bayesian inverse problem using multilevel delayed acceptance sampling. The underlying forward model involves a hierarchy of tsunami simulations run through ExaHyPE, with runtimes spanning several orders of magnitude and complex cross-level dependencies. Results demonstrate that the load balancer effectively handles such challenging workloads, making it a valuable tool for researchers in computational science and engineering who rely on UQ for simulation-based inference.
- Dynamic load balancer integrated into UM-Bridge, a language-agnostic UQ-simulation coupling framework.
- Achieves average node idle time of <1 ms without prior workload assumptions.
- Tested on Bayesian inverse problem with multilevel tsunami simulations in ExaHyPE (runtimes vary by orders of magnitude).
Why It Matters
Enables efficient HPC scheduling for complex UQ workflows, reducing wasted compute resources in simulation-based inference.