New paper shows how adversarial observations threaten RL systems and how to defend them

A new preprint from researchers M. Santos-Pascual and D. Ríos Insua tackles a critical vulnerability in reinforcement learning (RL) systems: adversarial attacks on the observation stream. The paper, titled "Adversarial observations in probabilistic State-Space Models for robust Reinforcement Learning," focuses on linear probabilistic state-space models (SSMs) that are commonly used in RL architectures for latent state inference. The authors model an attacker who subtly perturbs observations while staying within likelihood constraints to avoid detection. These adversarial perturbations propagate through the SSM, corrupting the estimated latent state and ultimately misleading the RL policy. The analysis reveals how even small, strategically crafted observation shifts can cause significant degradation in decision quality.

The work is particularly relevant for safety-critical applications such as robotics, autonomous driving, and industrial control, where sensors are prone to noise, partial failures, or deliberate tampering. By formalizing the attack surface and its impact on both inference and control, the authors lay out a principled pathway toward designing RL systems that are robust to adversarial observations. The paper includes 29 pages, 7 figures, and is currently a preprint undergoing ongoing work. It bridges techniques from statistical machine learning, control theory, and adversarial robustness, offering a unified framework that could inform future defense mechanisms.