Interpretable Deep Reinforcement Learning for Element-level Bridge Life-cycle Optimization

Researchers Seyyed Amirhossein Moayyedi and David Y. Yang have introduced a novel interpretable reinforcement learning (RL) framework designed to tackle a critical problem in civil infrastructure: optimizing bridge maintenance over its entire life-cycle. The challenge stems from new federal specifications (SNBI 2022) that require element-level condition data, shifting from a single rating to a complex four-dimensional probability array for each bridge. This explosion in state space makes traditional optimization methods impractical. The team's solution is an RL algorithm that produces its final policy not as a 'black box' neural network, but as a human-readable oblique decision tree.

The core innovation lies in three technical improvements to existing RL methods. First, they use differentiable soft tree models as the actor function approximator during training, which allows gradient-based optimization of a tree-like structure. Second, they implement a temperature annealing process to balance exploration and exploitation effectively. Third, they apply regularization paired with pruning rules to control the final policy's complexity. Collectively, these techniques enable the AI to learn a near-optimal maintenance strategy and then express it as a deterministic tree with a reasonable number of nodes, making it directly auditable by engineers.

The practical impact is significant for bridge management agencies. Instead of receiving an inscrutable AI recommendation, engineers get a clear set of interpretable rules—a decision tree—that specifies maintenance actions based on quantifiable bridge element conditions. This transparency builds trust and facilitates implementation into legacy management systems. The framework was successfully demonstrated on a life-cycle optimization problem for steel girder bridges, proving its utility for real-world, safety-critical infrastructure planning where explainability is non-negotiable.