Study: 68% of AI-generated webpages have cross-browser rendering failures

Multimodal large language models (MLLMs) are increasingly used to generate webpages directly from visual designs, but existing evaluations only check visual fidelity on a fixed browser-device configuration. This new study, authored by Ziyun Guo and colleagues, presents the first systematic empirical analysis of cross-environment rendering compatibility for AI-generated front-end code. The researchers constructed WebCompat, a dataset containing 2,032 annotated instances of webpages generated by 8 representative AI tools, each rendered across 9 browser-and-device combinations. The results are stark: 68% of generated webpages exhibit at least one compatibility issue. The most common symptoms are catastrophic layout failures (88.3%), where pages shrink to fit the target screen with overly small fonts or scale mismatches that cut off content. Issues localized to individual elements, such as image distortion or missing components, are less frequent (13.4%). The study also finds that while most MLLMs incorporate responsive design patterns, they frequently fail to implement these correctly, producing code that is semantically misleading.

To address these issues, the authors developed XCompat, a lightweight offline compatibility issue detector that analyzes both visual screenshots and the structural DOM tree. XCompat achieves an F1 score of 0.903 on the WebCompat-test set, outperforming existing compatibility checking tools and LLM baselines. This tool provides a practical way for developers to catch rendering bugs before deployment, without needing expensive cloud-based testing. All datasets and tools are publicly released to support future research on rendering reliability in MLLM-based code generation. The findings highlight a critical gap in AI-assisted web development: generating visually accurate code is not enough—it must actually work across the fragmented landscape of browsers, devices, and screen sizes. As more teams rely on LLMs to produce front-end code, tools like XCompat could become essential for maintaining quality and user experience.