New RL method for LQR cuts filter redundancy in MIMO control
Researchers reduce unknown parameters by proving only n(m+1) filtered components matter
Reinforcement learning for linear quadratic regulation (LQR) typically assumes full state feedback, but real systems often only provide input-output measurements. Chen et al. address this with a model-free output feedback framework that uses filtered input-output data instead of the state. The catch: existing filter-based parametrizations can contain redundant components, making Bellman regressions rank deficient. The authors formally characterize that intrinsic dimension, showing a SIMO system needs only 2n independent filtered components, while a general MIMO system needs just n(m+1), where n is the state dimension and m the input dimension.
Based on this insight, they construct a reduced filtered vector extracted directly from data, then derive reduced model-free policy iteration and value iteration equations. This eliminates redundant directions, shrinks the number of unknown parameters, and preserves a fully data-driven implementation. A numerical example verifies the rank reduction and shows the learned controller performs effectively. The result could significantly cut sample complexity and computational cost for applying RL to real-world MIMO control problems, from robotics to process control.
- Proves filtered state parametrization has only 2n independent components for SIMO and n(m+1) for MIMO systems
- Develops reduced model-free policy iteration and value iteration equations that remove redundant dimensions
- Numerical experiments confirm rank reduction and effective learned controllers with fully input-output data
Why It Matters
This makes model-free RL control more sample-efficient and practical for real MIMO systems with partial state observation.