Research & Papers

New benchmark fixes flaw in dynamic multi-objective optimization

Researchers propose benchmark where objective functions stay fixed while number changes.

Deep Dive

Dynamic multi-objective optimization with a changing number of objectives is critical for real-world problems where evaluation criteria evolve (e.g., in autonomous driving or portfolio management). However, existing benchmark test suites suffer from a fundamental flaw: when the number of objectives changes, the objective functions themselves also change implicitly. This makes it impossible to isolate an algorithm's capability to handle the dynamics of objective count alone, as performance differences could be due to function changes rather than count dynamics.

To address this, the researchers propose a new scalable benchmark suite where objective functions are fixed throughout optimization, while only the number of active objectives changes over time. They construct the suite by defining a maximum-objective problem and dynamically selecting subsets. To avoid degeneracy issues in classical DTLZ and WFG problems, they adopt Minus-DTLZ and Minus-WFG formulations, where all objectives are mutually conflicting. Extensive benchmarking with representative algorithms demonstrates the suite's usefulness and flexibility. This work provides a more rigorous tool for future research in this area.

Key Points
  • Existing benchmarks implicitly change objective functions when the number of objectives changes, confounding algorithm evaluation.
  • New benchmark fixes objective functions and only toggles the active subset of objectives over time.
  • Uses Minus-DTLZ and Minus-WFG formulations to ensure all objectives are mutually conflicting and avoid degeneracy.

Why It Matters

Enables fairer comparison of algorithms for real-world problems where evaluation criteria evolve over time.