Towards Scalable Lightweight GUI Agents via Multi-role Orchestration

A research team from multiple institutions has introduced LAMO (Lightweight Agent with Multi-role Orchestration), a novel framework designed to overcome the deployment bottleneck for GUI automation agents on consumer devices. Current state-of-the-art methods rely on massive multimodal LLMs that are prohibitively expensive to run locally, limiting practical automation to cloud-based solutions. LAMO addresses this by enabling a compact 3-billion parameter model (LAMO-3B) to perform complex desktop automation through intelligent role distribution, where the agent can adopt specialized personas like "navigator" or "executor" to tackle different aspects of a workflow.

The framework employs a sophisticated two-stage training regimen. First, it uses supervised fine-tuning with Perplexity-Weighted Cross-Entropy optimization to distill GUI-specific knowledge and enhance visual perception from larger teacher models. Second, it applies reinforcement learning for role-oriented cooperative exploration, teaching the agent to coordinate its internal roles effectively. This approach allows LAMO-3B to function both as a standalone agent and as a plug-and-play policy executor within larger multi-agent systems, meaning it can leverage advances in external planners without requiring complete retraining.

Extensive evaluations demonstrate that LAMO-3B achieves a compelling trade-off between capability and resource consumption. The research, accepted at ACL 2026, represents a significant step toward democratizing AI-powered desktop automation by making it feasible on standard laptops and mobile devices without constant cloud dependency. The architecture's flexibility suggests a future where lightweight agents can handle increasingly complex real-world software interactions.