Alibaba's Qwen-UI-Agent tops mobile UI benchmark with 92.2%

Alibaba's MAI-UI team has published a technical report on Qwen-UI-Agent, claiming a 92.2% success rate on MobileWorld-Real—a newly introduced benchmark that tests 409 tasks across 104 apps on over 100 physical Android devices. This performance is notable because most GUI-agent evaluations rely on emulators or simulated environments, which don’t capture real-world challenges like CAPTCHAs, expired logins, or unexpected pop-ups. The model outperforms rival agents by significant margins on this benchmark: 7.5 points ahead of Claude Opus 4.8, 6.8 points ahead of GPT-5.6 Sol, and 6.0 points ahead of Gemini 3.1 Pro.

Qwen-UI-Agent is available in three sizes: a 27B dense model, a 35B-A3B mixture-of-experts (MoE) with only 3B active parameters per token, and a lightweight 4B variant. The system unifies GUI interaction and bash CLI commands into a single action space, and over 40% of actions are batched to reduce unnecessary inference steps. Training involved online reinforcement learning across up to 10,000 concurrent environments and trajectories exceeding 100 turns, plus an agent-driven data flywheel for task construction and failure diagnosis. However, the benchmark and grading system (AutoJudge) are internally developed, and independent verification is pending. The 4B variant stands out for its practicality—it can run on ordinary hardware, enabling on-device deployment and empowering small teams to build advanced agents without reliance on costly cloud APIs.