Metaverse gesture framework uses MediaPipe for hands-free avatar navigation

Researchers Deepti Parachuri, Chhayank Sahu, and Sameer Singh Choudhary have published a framework that brings gesture-driven avatar control to browser-based metaverse onboarding environments. The system replaces traditional keyboard or controller inputs with real-time hand, arm, and head gestures, recognized via Google MediaPipe. Users explore virtual rooms and interact with embedded video, document, and quiz content without touching a single button. The core innovation is combining two locomotion techniques—hand-raise navigation and in-place walking—within one lightweight, web-deployable layer. This lets users choose between precision-oriented and physically immersive movement styles depending on the task, a trade-off the authors explicitly evaluate.

The framework was assessed through a structured internal onboarding session with five participants, documenting what worked and what didn't in real deployment. The paper reports design trade-offs that emerged when blending gesture recognition with different movement mechanics, such as balancing accuracy against physical effort. The authors position this as an integration and evaluation contribution rather than a new algorithm—proving that controller-free avatar interaction is feasible in a standard web browser. While the sample size is small, the insights around usability and immersion are valuable for metaverse platform builders aiming to lower onboarding friction and make virtual spaces more accessible to non-technical users.