Researchers tighten average-reward RL regret bounds by 94%

Evaluating regret bounds in average-reward reinforcement learning is notoriously messy: papers use different probability modes, structural assumptions, and normalization, making apples-to-apples comparison nearly impossible. The authors tackle this head-on with a constant-aware comparison protocol and derive an explicit finite lower certificate for communicating MDPs. Their construction uses a binary tree of two-state blocks and exact trajectory-level KL divergence, keeping action budget, diameter, and occupancy costs explicit. The result: the published coefficient improves from 0.015 to 0.0200 in a moderate regime, and up to 0.0291 under stronger assumptions—a 94% increase. The limiting coefficient is (1/32)√((A-3)/A), a clean closed form that encodes action-space dependence.

On the upper-bound side, the paper offers an auditable composition rule for a span-constrained optimistic learner, though it stops short of claiming a final coefficient because adaptive directional variance and planning certificates remain open. It also formalizes valid expectation conversion and constant comparability, giving the field a much-needed standard for future work. Controlled diagnostics test diameter dependence, bonus-by-width interactions, and span misspecification, ensuring the lower certificate behaves correctly on its exact family. For RL researchers and practitioners, this means more trustworthy benchmarks and clearer guidance on when tighter guarantees actually apply—bringing theoretical rigor one step closer to deployable decision-making systems.