SPRITE: From Static Mockups to Engine-Ready Game UI

A research team including Yunshu Bai, RuiHao Li, and Ming Yan has introduced SPRITE, a novel AI pipeline designed to solve a persistent bottleneck in game development: translating static UI mockups into interactive, engine-ready code. Current "screenshot-to-code" tools often fail with the irregular geometries and deep visual hierarchies typical of stylized game interfaces. SPRITE bridges this gap by integrating Vision-Language Models (VLMs) with a structured YAML intermediate representation, which explicitly captures these complex container relationships and non-rectangular layouts that standard web-focused tools miss.

The system was rigorously evaluated against a curated Game UI benchmark and reviewed by professional developers to assess both reconstruction fidelity and prototyping efficiency. The findings, accepted for CHI EA '26, demonstrate that SPRITE successfully automates the tedious, manual coding required to rebuild mockups in engines like Unity or Unreal. By generating editable assets and facilitating rapid in-engine iteration, the tool effectively blurs the traditional boundaries between artistic design and technical implementation, potentially cutting days off UI prototyping cycles.