OpeFlo: Automated UX Evaluation via Simulated Human Web Interaction with GUI Grounding

A research team including Wee Joe Tan, Zi Rui Lucas Lim, and Aiden Yiliu Li has developed OpenFlo, a novel AI system that automates user experience (UX) evaluation by simulating human interaction with websites. Unlike traditional tools that rely on parsing a website's Document Object Model (DOM), OpenFlo uses GUI grounding—meaning it processes and interacts with the visual interface as a human would, through screenshots or similar visual inputs. This approach allows it to navigate real, live web pages end-to-end, maintaining a coherent trace of the simulated user journey. The system is built upon Avenir-Web, providing a robust foundation for these visual interactions.

OpenFlo pairs this visual interaction capability with simulated user behavior profiles and a structured evaluation protocol. It integrates established UX metrics: the System Usability Scale (SUS) for an overall score, step-wise Single Ease Questions (SEQ) for task-specific feedback, and concurrent Think Aloud for qualitative insights. After its simulated session, the agent automatically compiles a comprehensive UX report. The researchers argue that this multimodal, grounded approach significantly improves robustness for automated web interaction and evaluation, moving beyond the limitations of DOM-based scripts that can break with layout changes.

This development paves the way for a new paradigm in usability testing. By automating a process that typically requires time-consuming, expensive user studies or expert reviews, OpenFlo aims to make continuous, data-driven UX evaluation scalable. This is particularly impactful for small development teams and agile workflows where rapid iteration is crucial. The code for OpenFlo is publicly available, inviting further development and application in the field of AI-assisted human-computer interaction.