Lumosaic: Hyperspectral Video via Active Illumination and Coded-Exposure Pixels

A research team from the University of Toronto and University of Washington has developed Lumosaic, a breakthrough hyperspectral imaging system that captures real-time spectral video of dynamic scenes. Unlike traditional snapshot hyperspectral systems that struggle with motion artifacts, Lumosaic actively synchronizes a narrowband LED array with a coded-exposure-pixel (CEP) camera, enabling joint encoding of spatial, temporal, and spectral information within each frame. This approach significantly improves photon utilization and preserves spectral accuracy even during rapid movement, addressing a fundamental limitation of passive systems that divide light across spectral channels and assume static scenes during exposure.

The system's technical innovation lies in its hardware-software co-design, where the CEP camera provides per-pixel exposure control at high speeds while the LED array sequentially illuminates the scene with different wavelengths. A neural network-based reconstruction pipeline then processes this encoded data to recover 31-channel hyperspectral video (400-700 nm) at 30 frames per second with VGA resolution. Experiments demonstrate that Lumosaic produces temporally coherent and spectrally accurate reconstructions across diverse materials and motion conditions, outperforming existing snapshot systems in both reconstruction fidelity and temporal stability. The work, accepted to CVPR 2026, represents a significant advancement toward practical hyperspectral video applications in fields requiring real-time spectral analysis of moving objects.