Researchers build HoloLens 2 edge rendering system with AI-based quality metric
New testbed dynamically switches XR rendering between headset and edge server for low latency.
Extended Reality (XR) applications demand heavy computation and ultra-low latency, making local-only rendering often infeasible. Researchers from (the authors' institution) introduce a testbed that dynamically offloads rendering workloads between a Microsoft HoloLens 2 headset and a GPU-enabled edge server using the HOLO Stream SDK. The system monitors network conditions and latency in real time, seamlessly switching between local and edge rendering modes. This hybrid approach balances computational load while maintaining responsiveness.
To address the failure of pixel-level metrics under head movements and asynchronous frame arrivals, the team developed a perceptual evaluation metric based on deep feature embeddings and cosine similarity. This metric remains robust to spatial and temporal misalignments. Additionally, they designed a contextual bandit learning controller that adapts rendering placement decisions by jointly optimizing perceptual quality and latency. Experimental results demonstrate the testbed's feasibility, delivering high-quality interactive XR experiences while keeping latency low. The work was published on arXiv in June 2026.
- Testbed uses Microsoft HoloLens 2, GPU edge server, and HOLO Stream SDK for real-time switching between local and edge rendering.
- Novel perceptual quality metric based on deep feature embeddings and cosine similarity handles spatial/temporal misalignments from head movements.
- Contextual bandit controller dynamically optimizes rendering placement for joint perceptual quality and latency improvement.
Why It Matters
Enables high-quality, low-latency XR experiences by intelligently splitting rendering between device and edge.