From Prompts to Worlds: How Users Iterate, Explore, and Make Sense of AI-Generated 3D Environments

A new study by researcher Aung Pyae provides the first in-depth look at how people actually interact with commercial text-to-3D generative AI systems. Unlike evaluating a static image, assessing a 3D environment requires users to navigate and explore it. The research, combining think-aloud protocols and behavioral observation, identifies a core problem: 'asymmetric expressibility.' Users can readily describe the semantic feel of a world—like a 'spooky forest' or 'futuristic city'—but their language fundamentally fails when trying to dictate precise spatial structures, layouts, and scale. This is a system limitation, not a user skill issue.

This language-to-space gap leads to a cycle of frustration. Users experience only 'episodic presence,' feeling immersed in moments when the output accidentally matches their mental image, but this never builds into a sustained illusion of being in a place. Attempts to refine the world through iteration often break down due to poor tool discoverability, opaque feedback from the AI, and the high time cost of regenerating complex 3D scenes. The study concludes that effective text-to-3D should be reframed as a 'negotiated meaning-making' process between user and AI, requiring future platforms to integrate hybrid inputs (like sketches or drag-and-drop), provide transparent feedback, and drastically lower the cost of iteration to enable true creative exploration.