CoGen3D pipeline lets non-experts co-create VR assets with AI guidance

CoGen3D, developed by a team including researchers from multiple institutions, addresses a key barrier in VR content creation: the need for 3D modeling expertise. The pipeline uses a conversational agent to elicit user intent through structured dialogue, then generates a concept image for confirmation before producing the final 3D asset. This staged approach replaces unconstrained prompting with guided co-design. In a study with 120 participants across six affectively diverse immersive scenes, the design group co-created assets while a validation group experienced the scenes. Results showed that co-designed assets led to higher scene engagement and shifted emotional responses. Interestingly, participants preferred concept images over final 3D assets, yet did not show increased leniency toward degradation in their own creations, indicating high-quality expectations.

The analysis of human-AI conversations revealed that target environments significantly shaped conversational patterns—users adapted their language and requests based on scene context. These findings suggest that intent-based, scaffolded co-design can effectively democratize VR authoring, shifting the role of creators from technical execution to collaborative spatial design. CoGen3D represents a notable step toward making immersive content creation accessible to non-specialists while maintaining quality control through human-in-the-loop validation. The system's agentic guidance reduces the cognitive load of translating vague ideas into precise 3D specifications, potentially lowering barriers for educators, storytellers, and industry professionals without 3D modeling backgrounds.