AppAgent-Claw turns GUI workflows into reusable CLI skills without LLM inference

A new paper by Zhixue Song and colleagues presents AppAgent-Claw, a demonstration-driven system that bridges the gap between GUI-heavy tasks and the CLI-centric OpenClaw automation platform. Current LLM-based GUI agents require repeated live model inference, making them too slow, expensive, and inconsistent for practical use as OpenClaw skills. AppAgent-Claw solves this by adopting a 'record-once, replay-many' paradigm: users demonstrate a GUI workflow once, and the system captures rich contextual metadata to enable robust replay without any runtime AI inference.

The system's technical core includes a layered localization strategy that adapts to visual layout changes (e.g., window resizing or theme shifts) and a validation-coupled execution model that verifies each step's on-screen effect before proceeding. This ensures reliable, repeatable automation for tasks that lack stable APIs. AppAgent-Claw is designed to slot into the OpenClaw ecosystem as a skill, providing a practical, efficient, and diagnosable solution for GUI-bound tasks—without sacrificing the generality that LLM agents offer. The paper includes 13 pages, 1 figure, and 8 tables of benchmarks.