VISTA Benchmark Tests AI's Ability to Build Web Apps from Visual Specs

VISTA (VIsual Spec-To-App Benchmark) addresses a critical gap in code generation evaluation by focusing on realistic, UI-centric web application development. Unlike existing benchmarks that test algorithmic problem-solving, VISTA requires AI agents to produce functional, visually coherent applications from underspecified inputs. The benchmark defines five prompt-information conditions that vary along two axes: visual/structural fidelity (text only, text + screenshots, text + screenshots + pruned Figma structure) and stack constraints (free choice vs. single specified stack). This allows researchers to systematically measure how different levels of input detail affect agent performance.

To enable robust evaluation, authors manually annotated each page with interactive UI components and around three visual anchor points, overcoming limitations of script-based testing tools like Playwright in open-ended code generation. The evaluation combines three complementary metrics: DOM-grounded reference matching (structural alignment), behavior-specific browser tests (functional correctness), and CLIP-based visual similarity (overall visual fidelity). Testing four agent systems from two model families and two harnesses, the study found that visual fidelity and functional correctness are partially decoupled across input conditions and agents. Additionally, agent editing style varies sharply but is largely orthogonal to task quality, suggesting that current agents lack consistent strategies for balancing aesthetics and functionality.