Bastani et al. formalize policy learning hierarchy with three problems

A new paper from researchers Hamsa Bastani, Osbert Bastani, and Shihan Chen (arXiv:2607.03385) tackles a fundamental question in policy learning: what can we do when data is too scarce to find the optimal policy? The authors propose a mathematical framework that defines three distinct policy learning problems: the optimal policy problem (minimize regret), the improving policy problem (find a policy that statistically outperforms baseline), and a new 'policy existence problem' (determine if any improving policy exists at all). They prove that each problem reduces to the next in terms of sample complexity — existence is at least as easy as improving, which is at least as easy as optimal.

Crucially, the authors demonstrate that the gap between optimal and improving is strict, and they provide partial proofs that a sublinear polynomial gap exists between improving and existence under natural conditions. This means that with limited data, we may be able to answer whether a better policy exists without being able to find it — a practical boon for high-stakes domains like healthcare or finance where data is expensive or limited. The work shifts focus from always needing to find the best policy to asking more nuanced questions about what is knowable with available data.