Reimagining Wearable AR Gesture Design: Physical Therapy Reasoning in Everyday Contexts

A research team from Northeastern University and UC San Diego has published a groundbreaking paper for CHI 2026 that fundamentally rethinks how we design gestures for everyday augmented reality (AR) glasses. The core insight is that current gesture vocabularies, often inherited from VR or early AR, prioritize recognizer accuracy while ignoring long-term user fatigue, sustainability, and social acceptability. To solve this, the researchers uniquely partnered with domain experts in sustainable human movement: physical therapists. Through a review of 104 existing AR applications, they first identified 15 common user intents (like 'select' or 'scroll') that need gesture mappings.

The team then engaged ten licensed physical therapists, each with an average of nearly 15 years of clinical experience, in a rigorous, three-stage design process. This included unaided gesture performance, therapist-guided substitutions for problematic motions, and a novel 'stage-aware' card sorting activity. The collaboration yielded three key contributions: a PT-informed method for translating digital intents into body-friendly motions, the 'Everyday-AR Golden Ergonomic Canvas' as a design tool, and a framework showing how a gesture's social legibility changes in different contexts (private vs. public). Together, this provides a recognizer-agnostic blueprint for creating AR interactions that are ergonomically sound and socially coherent for all-day use.