This AI Explorers Virtual Worlds to Find Bugs Before You Do

Testing virtual reality and 3D apps is notoriously tricky. If a game or training simulator has a bug, it might only appear when you stand in a certain spot, look at a certain object, or capture a specific view. That's why developers still rely on humans to wander through virtual worlds and report what breaks — a slow, expensive process that's hard to repeat.

ExploreAI, a new system from researchers, replaces that human wanderer with artificial intelligence. It uses a large language model (a smart AI that understands language and makes decisions) to do what a human tester does: figure out what the task is, choose which objects to inspect, group related things, and decide if it needs to look again when something seems missing. Meanwhile, specialized modules handle the boring, repeated work of moving, looking, and taking screenshots.

What makes ExploreAI special is that it keeps a detailed diary of everything it does. Called an Exploration Knowledge Base, this diary records the path it took, the views it captured, and whether it verified each object properly. When a developer updates a VR app, they can use that diary to re-check the same exact views and see if anything broke — basically, a reproducible bug test that anyone can rerun.

The team tested ExploreAI in six scenes across three platforms, including Unity, AI2-THOR, and a driving simulator. It built highly complete exploration records, and the traces it created even helped human testers reproduce findings more accurately. So while you won't see ExploreAI on your headset, it could be working behind the scenes to make sure your next virtual adventure doesn't glitch out mid-game.