VisionClaw: Always-On AI Agents through Smart Glasses

A research team including Xiaoan Liu, DaeHo Lee, and Eric J. Gonzalez has developed VisionClaw, a novel AI agent system that transforms Meta Ray-Ban smart glasses into always-on wearable assistants. The system continuously perceives the user's real-world environment through the glasses' cameras and enables in-situ, speech-driven task execution via integration with OpenClaw AI agents. This allows users to perform actions like adding physical objects to an Amazon cart, generating notes from documents, receiving meeting briefings, creating calendar events from posters, or controlling IoT devices—all through voice commands while wearing the glasses.

In their evaluation, the researchers conducted both a controlled laboratory study with 12 participants and a longitudinal deployment study with 5 users. Results showed that VisionClaw's integration of continuous perception with agentic execution enabled significantly faster task completion and reduced interaction overhead compared to non-always-on and non-agent baselines. Beyond performance metrics, the deployment study revealed a fundamental shift in interaction patterns: users began initiating tasks opportunistically during ongoing activities rather than in dedicated sessions, and increasingly delegated execution to the AI agents rather than maintaining manual control.

The researchers argue that VisionClaw represents a new paradigm for wearable AI where perception and action are continuously coupled to support situated, hands-free interaction. The system, detailed in a 10-page paper submitted to UIST 2026, suggests a future where smart glasses become proactive assistants that understand context and execute tasks seamlessly in real-world environments. This approach moves beyond simple voice commands to create true agentic systems that can perceive, reason, and act on behalf of users in their daily lives.