ComAct uses COM to let AI control professional software with near-zero GUI failure

Existing AI agents for professional software manipulation are fundamentally limited: GUI-based agents suffer fragile visual grounding and long-horizon error accumulation, while API-based agents struggle with heterogeneous protocols and inaccessible commercial interfaces. ComAct reframes the problem by leveraging the Component Object Model (COM) — a unified executable abstraction already present in many Windows applications — turning software interaction from sequential visual control into deterministic program synthesis. This paradigm shift bypasses pixel-level fragility and allows agents to directly call software functions.

To validate this in the most demanding environments, the researchers created ComCADBench, the first benchmark for agents operating real industrial CAD software. Experiments show a substantial paradigm gap: frontier proprietary models achieve near-zero success under GUI interaction, whereas COM-based execution yields large immediate gains. To bridge the remaining gap between syntactic correctness and geometric accuracy, they developed ComActor, a self-correcting agent trained through a progressive three-stage framework, and ComForge, a scalable training platform using Windows containers. ComActor achieves state-of-the-art performance on ComCADBench, with strong resilience in long-horizon tasks where baselines collapse, and generalizes to external CAD benchmarks.