Your AR Glasses Could Get Smarter — Thanks to a New Test Track

Headsets and smart glasses are supposed to show you the right button at the right moment — start a video call, pull up directions, mute a noisy room — without you hunting for it. In practice, you swipe through layers of menus while the moment passes. The hard part isn't building the features. It's proving whether one design guesses your needs better than another.

A team of researchers from Carnegie Mellon, Meta and the University of Washington built a way to settle that argument. Their system, ContextXR, treats an app as a connected map of "functional facets" — plain-English groups of related tools that serve one goal, like everything you'd need to join a meeting. They also released MineXR++, a dataset of real headset interfaces labelled with these groups, and defined three standard tasks: figuring out which context clues matter, suggesting the first tool, and suggesting the next one.

To judge the suggestions, they used a simulated "navigation cost" — basically counting the taps and menu dives you'd need to reach what you actually wanted. Lower cost, better design. They then compared three approaches: showing whatever is most popular overall, showing things related to what you just did, and asking an AI model to predict your next move. Because everyone now measures the same way, results become comparable across studies and users.

The honest catch: this is a simulation, not real people wearing headsets, and it's a research framework rather than a product you can buy. No headset maker has shipped it yet. But it's the kind of unglamorous plumbing that tends to show up later as smoother, less annoying software — the scorecard that tells companies which of their clever ideas actually helps you.