New ML algorithm outperforms mean-based bandits in arm selection

Researchers Jonghyun Sim and Wonyoung Kim from South Korea have published a groundbreaking paper on arXiv (arXiv:2608.01545) that redefines how multi-armed bandit problems are solved. Their work, titled 'Dominant Arm Identification with Mixing and Recycling Observed Samples,' addresses a critical flaw in conventional approaches: mean-based algorithms often fail to identify the arm with the highest realized reward, even when it exists.

The core innovation lies in two technical breakthroughs: a dominance score criterion that evaluates arms based on their performance relative to others in partitioned reward spaces, and a joint mixing-and-recycling mechanism paired with a doubly robust estimator. Together, these innovations ensure simultaneous convergence of empirical distribution functions for all arms. The proposed elimination algorithm demonstrates nearly optimal sample complexity and achieves exact recovery of the true dominant arm in numerical experiments, outperforming existing baselines. This represents a significant leap for decision-making systems in fields like recommendation engines, clinical trials, and resource allocation.