VR + LLMs: Spatial previews slash confusion in AI-assisted geometry editing

A team of researchers led by Junlong Chen (University of Cambridge) and colleagues from Coburg University and the University of Cambridge has published a paper on arXiv exploring how intent disambiguation can be improved for LLM-assisted geometry editing in virtual reality. The paper, titled "Beyond Conversations: Spatially-Anchored Previews for Intent Disambiguation in LLM-Assisted Geometry Editing in Virtual Reality," addresses a key challenge in immersive environments: when users issue ambiguous commands to an LLM, traditional 2D dialogue-based clarification can break the spatial context that makes VR powerful. The researchers propose and evaluate a hybrid disambiguation approach that augments LLM clarification questions with spatially-anchored graphical previews—visual overlays that show exactly what the LLM interprets the user's intent to be, directly in the 3D scene.

In a within-subjects study with 24 participants performing complex parameter-driven geometry editing tasks in VR, the hybrid method was compared against a baseline with no disambiguation. Quantitative metrics showed that the hybrid approach significantly reduced the number of conversation rounds needed to resolve ambiguity, while qualitative feedback indicated higher user satisfaction and perceived interaction stability. The study provides empirical evidence that combining spatial visual hints with conversational clarification is more effective than either alone for LLM editing in VR. The authors distill their results into design guidelines for future VR/AR systems that integrate large language models, emphasizing that spatially-anchored previews can help bridge the gap between human intent and machine interpretation in immersive 3D workflows.