New benchmark exposes gaps in AI mobile app generation

A team of researchers from six universities—including the University of Hong Kong and Harbin Institute of Technology—has developed **MobileForge**, the first benchmark designed to evaluate how well multimodal large language models (LLMs) can generate complete, multi-screen mobile applications from visual designs. Published on arXiv as *cs.HC* (arXiv:2607.28645), the work focuses on project-level generation, where apps require multiple interconnected screens, shared components, and functional navigation—far beyond the single-page capabilities tested by existing benchmarks.

The benchmark introduces a five-axis evaluation system: build success, navigation reliability, visual fidelity, code maintainability, and efficiency. It includes real mobile apps with human-reviewed screens, structured page relationships, and test specifications. Early tests on six frontier multimodal LLMs (not named) revealed that while models can generate code that compiles and reaches the correct pages, interactive navigation remains unreliable. Visual fidelity and code maintainability also lag significantly, highlighting critical gaps in current AI app development tools.