New Framework Optimizes Restless Bandits with Imperfect Feedback
Whittle index policy outperforms benchmarks in spectrum access with sensing errors.
Restless bandits are a powerful model for sequential resource allocation problems with uncertainty, such as cognitive radio spectrum access where channels are unavailable or idle. However, existing methods often assume perfect feedback. A new paper by José Niño-Mora tackles the realistic scenario of imperfect binary feedback (e.g., sensing errors) with binary latent states. The paper develops a comprehensive Partial Conservation Laws (PCL) framework to analyze and compute the Whittle index, which yields a near-optimal policy.
The framework uses a deterministic skeleton and combinatorial analysis to derive tractable expressions for discounted reward and resource metrics in several threshold regimes, fully verifying PCL-indexability there. For regimes where analytic verification is incomplete, efficient numerical schemes are derived to compute marginal metrics and the marginal productivity (MP) index. Extensive computational experiments across broad parameter ranges provide strong evidence that the indexability conditions hold, and the MP index policy typically outperforms standard benchmark policies by a substantial margin.
- The paper addresses restless bandits with binary latent states and imperfect binary feedback, motivated by spectrum access.
- A PCL-based analytical and computational framework is developed, verifying indexability in many threshold regimes.
- Experiments show the MP index policy typically outperforms standard benchmarks, often by a substantial margin.
Why It Matters
Enables near-optimal resource allocation algorithms for real-world systems with sensing errors, improving efficiency in spectrum, ads, and more.