New AI control algorithm skips critic, slashing data needs for LQR

In reinforcement learning for control, policy iteration traditionally alternates between a critic—estimating the value function—and an actor that improves the policy. For continuous-time linear quadratic regulation (LQR) with unknown system dynamics, that critic step typically solves a joint regression that scales as O(n²) in the state dimension n. A new paper from Jiacheng Wu, Yang Zhu, and Hongye Su (arXiv:2607.08204) challenges this orthodoxy. They show that the critic is unnecessary: by anchoring the Riccati equation at a known stabilizing gain, they express optimality as a policy-space residual. An endpoint null-space projection then eliminates the value-matrix term from the integral data equation, yielding a direct, actor-only least-squares update computed solely from input-state data.

Under a verifiable projected rank condition, this critic-free update coincides exactly with the Kleinman iteration, meaning it inherits the same stabilizing and convergence properties. The authors also decompose the conventional full-rank condition into two parts: an endpoint critic rank condition and a projected actor rank condition. Their method removes the critic rank requirement while keeping the one needed for policy improvement. The computational savings are significant: the repeated least-squares dimension drops from n(n+1)/2+mn to just mn, where m is the input dimension. Comparative simulations confirm the algorithm works as well as standard off-policy methods while using less data and computation. This opens the door to simpler, more efficient data-driven control for systems like robotics, autonomous vehicles, and industrial processes where system matrices are unknown.