ActFovea boosts VLA robot success from 49% to 90% under visual attacks

Vision-language-action (VLA) models like π0 have shown impressive dexterity in robotic manipulation, but they remain fragile when real-world disturbances break the alignment between what the robot sees, its internal state, and the actions it executes. A new paper from Wenda Yu and colleagues introduces ActFovea, a plug-and-play runtime safeguard that catches these failures without retraining or modifying the underlying policy. ActFovea fuses robot kinematics, proprioceptive states, and recent action history to build action-conditioned foveated regions, suppressing irrelevant visuals while keeping contact-relevant areas and predicted motion corridors in focus.

ActFovea detects risk by checking whether visual motion and observation freshness stay consistent with geometric, proprioceptive, and action transitions. If a disturbance is recoverable, it constructs candidate observations and validates the resulting action chunk before accepting recovery. When observations are stale or replayed beyond repair, it executes a bounded safe-failure procedure. In closed-loop tests on π0 across multiple LIBERO suites, ActFovea lifted success under visual overlays from 49.3% to 90.3%, closing 93.7% of the gap to clean performance. It also improved success under action drift by 7.0 percentage points and under visual delay by 9.8 points, all while preserving clean-task performance. Under frozen-observation replay, it triggered timely safe failure in every trial with no unprotected failures.