UI-Oceanus: Scaling GUI Agents with Synthetic Environmental Dynamics

A research team led by Mengzhou Wu and 18 collaborators has introduced UI-Oceanus, a novel framework designed to overcome the scalability limitations of current GUI agents. Traditional methods rely heavily on expensive human demonstrations or synthetic teacher supervision, hitting what the researchers term a "distillation ceiling." UI-Oceanus shifts the paradigm by focusing the agent's learning objective on mastering the underlying physics of user interfaces through forward dynamics prediction. Instead of simply mimicking high-level action sequences, the agent learns to generate predictions of future interface states based on its actions, using ground-truth environmental feedback from the system itself. This creates a robust internal world model from low-cost autonomous exploration.

Experimental results demonstrate the decisive superiority of this approach. Models trained with Continual Pre-Training (CPT) on synthetic dynamics data outperformed baseline models by an average of 7% on offline benchmarks. More impressively, this performance gap widened to a 16.8% gain in real-world online navigation tasks, showing significantly better cross-domain adaptability. The research also confirmed that navigation performance scales reliably with the volume of synthetic training data. By grounding the agent in forward predictive modeling, UI-Oceanus provides a more effective pathway to creating scalable GUI automation with strong compositional generalization, moving beyond the limitations of imitation learning.