New Prolog method turns black-box RL policies into explainable logic programs

Deep reinforcement learning policies are notoriously opaque, behaving as black boxes that resist human inspection. In a new paper, Eduardo C. Garrido-Merchán proposes a three-stage post-hoc transformation that rewrites a trained proximal policy optimization (PPO) network into an executable Prolog program. First, the neural teacher is frozen and its decisions are sampled. Second, an ordered rule list is induced from those decisions using classical relational learning techniques. Third, the rules are emitted as a Prolog program that an off-the-shelf logic engine can run, edit, and certify. The method proves four guarantees: a return-loss bound that makes the distilled program a machine-checkable certificate in finite Markov decision processes; monotonic improvement under rule editing; termination of the expansion loop; and for continuous observations, a conversion to arbitrary fidelity with disagreement O(1/B) as resolution B grows, though with exponential cost in observation dimension for oblique boundaries.

Empirically, the approach shines on discrete tasks. In a two-room key-and-door environment with 16,944 reachable states, the expanded Prolog program attained exact optimal return in every seed, and under a budget-capped regime it outperformed the stochastic teacher on exact return in ten of ten seeds. On continuous control, the emitted Prolog program substituted the neural network directly: matching the teacher within noise on Acrobot with just eleven clauses, recovering about 97% of its return on CartPole, but only partially on the finer-control LunarLander—exactly the ceiling predicted by the exponential lower bound. This work bridges deep RL with symbolic AI, offering a path to verifiable, human-readable policies without sacrificing performance.