OSGuard benchmark reveals safety gaps in computer-use AI agents

As AI agents increasingly handle desktop and web tasks autonomously, measuring success solely by task completion can mask dangerous shortcuts. Researchers Mina Mohammadmirzaei and Jeffrey Flanigan tackle this with OSGuard, a new benchmark suite designed to evaluate safety under benign, unchanged user instructions. OSGuard operates at two granularities: an action-level benchmark that classifies individual proposed actions as allowed, unrelated, or unsafe against the original instruction and interface state; and a risk-augmented execution suite derived from OSWorld tasks, where the environment is modified to introduce latent hazards like destructive overwrites while keeping the original goal achievable. Augmented evaluators add state-based safety invariants alongside task-success criteria to distinguish safe from unsafe completions.

Experimental results reveal a significant gap. Multimodal guardrails (e.g., CLIP-based or LLM-based) achieve strong performance on isolated action judgments—accurately flagging unsafe moves in individual steps. However, when tested end-to-end in the risk-augmented execution suite, these same models often fail to prevent unsafe completions that still reach the nominal goal. This exposes a critical weakness: local oversight does not guarantee safe full-task behavior. OSGuard enables precise diagnosis of whether models can both recognize unsafe actions and improve overall safety when deployed as guardrails, pushing the field toward more robust autonomous agents.