Spectrum-Aware AI Estimates Lighting Accurately from Multispectral Images

Multispectral (MS) imaging captures more spectral bands than conventional RGB, enabling better estimation of the scene's illuminant spectrum (ISE). But existing methods fail to fully exploit that spectral richness, especially under varied lighting and across different camera sensors. To fix this, researchers from Ewha Womans University and Inha University propose a deep learning framework that uses a spatio-spectral feature extraction block with spectral attention mechanisms. This block enhances spectral correlation while preserving illuminant-relevant spatial features.

A key innovation is the inclusion of an illuminant prior (IP), which helps the model focus on specific spectral channels that carry the most meaningful information. The team also introduces a spectral-domain transform that lets illuminant spectra learned in high-dimensional sensor spaces be transferred to various lower-dimensional camera sensor spaces—without any additional training. This means a single trained model can serve many different camera types, drastically reducing deployment costs.

To validate their method, the authors collected a new real-world MS dataset containing high-dimensional ground-truth illumination spectra captured under diverse lighting conditions. Extensive experiments show their approach achieves superior accuracy compared to existing models. The code and dataset are publicly available, and the paper has been accepted for publication in IEEE Transactions on Circuits and Systems for Video Technology (TCSVT).