On The Complexity of Best-Arm Identification in Non-Stationary Linear Bandits

A team of researchers from the University of Washington and beyond has published a significant paper tackling a core problem in reinforcement learning: efficiently identifying the best option (or 'arm') in a non-stationary environment with a fixed time budget. The paper, 'On The Complexity of Best-Arm Identification in Non-Stationary Linear Bandits,' addresses a scenario where the underlying reward parameters can change adversarially over time, a realistic model for many real-world applications like online advertising where user preferences shift. The authors first identified a key limitation in existing theory, showing that the standard G-optimal design approach provides a pessimistic, worst-case bound that doesn't account for the geometric structure of the available choices.

To solve this, the team established a new, tighter lower bound that depends on the specific set of arms. More importantly, they developed a novel algorithm called Adjacent-BAI, which is a specialization of the XY-optimal design. They proved that Adjacent-BAI's performance matches their new theoretical lower bound up to constant factors, demonstrating that their bound is tight and that their algorithm is essentially optimal for this problem class. This work provides both a deeper theoretical understanding of the problem's complexity and a practical, provably optimal algorithm for making sequential decisions under changing conditions with limited exploration time.