Pareto-Optimal Anytime Algorithms via Bayesian Racing

A team of researchers from the University of Augsburg and collaborating institutions has introduced a novel framework called PolarBear (Pareto-optimal anytime algorithms via Bayesian racing) that fundamentally changes how optimization algorithms are benchmarked and compared. The core innovation addresses a critical limitation in current evaluation methods: the computational budget for real-world deployment is often unknown during benchmarking. Traditional approaches either collapse performance into a single scalar metric, require manual interpretation of complex plots, or produce conclusions that change when algorithms are added or removed. These methods also typically rely on normalized objective values, which require problem bounds or optima that are frequently unavailable and break coherent aggregation across different problem instances.

PolarBear formulates algorithm comparison as a Pareto optimization problem over time, where an algorithm is considered non-dominated if no competitor beats it at every timepoint. By using rankings rather than raw objective values, the approach eliminates the need for normalization, bounds, or known optima. The framework employs Bayesian inference over a temporal Plackett-Luce ranking model to provide posterior beliefs about pairwise dominance, enabling early elimination of confidently dominated algorithms through adaptive sampling with calibrated uncertainty. This results in significant computational savings during the benchmarking process.

The output of PolarBear is a Pareto set of algorithms along with posterior distributions that directly support downstream algorithm selection. This allows practitioners to choose algorithms based on their specific time preferences and risk profiles without requiring additional experiments. The method's 32-page paper, submitted to ACM Transactions on Evolutionary Learning and Optimization, demonstrates the framework through 12 figures and 2 tables, showing how it maintains coherent aggregation across arbitrary instance distributions. This represents a substantial advancement in meta-algorithmic research, providing a more robust foundation for comparing optimization methods in real-world scenarios where computational budgets are uncertain.