Beyond Hard Constraints: Budget-Conditioned Reachability For Safe Offline Reinforcement Learning

Researchers Janaka Brahmanage and Akshat Kumar have introduced a novel algorithm for safe offline reinforcement learning (RL) titled 'Budget-Conditioned Reachability.' The core innovation addresses a critical flaw in existing methods: most safety-focused RL techniques handle only hard constraints (e.g., "never crash") or rely on unstable adversarial optimization to balance rewards with safety. This new work shifts the paradigm by first precomputing a 'safety-conditioned reachability set'—a forward-invariant zone of safe states and actions. This decouples the problem, allowing an agent to maximize reward within a pre-defined safety budget without the min/max instability of Lagrangian methods.

The algorithm is designed for the offline RL setting, meaning it learns safe policies solely from a fixed historical dataset, eliminating the need for risky or expensive live environment interaction. This is crucial for real-world applications like robotics and autonomous systems, where trial-and-error learning is impractical. The authors validated their approach on standard offline safe RL benchmarks and a practical maritime navigation task. Results demonstrated that the method not only maintains safety but also matches or exceeds the performance of current state-of-the-art baselines. The work was accepted for presentation at the 36th International Conference on Automated Planning and Scheduling (ICAPS 2026).