Unlocking Multi-Spectral Data for Multi-Modal Models with Guided Inputs and Chain-of-Thought Reasoning

A team of researchers from Google, including Dahun Kim, Ganesh Satish Mallya, and Anelia Angelova, has introduced a novel training-free approach that enables standard RGB-only large multi-modal models (LMMs) to effectively process multi-spectral imagery. Their method, detailed in a paper accepted to IGARSS 2026, addresses a critical limitation: while multi-spectral data is essential for remote sensing tasks like land-use classification and environmental monitoring, generalist LMMs are typically trained exclusively on RGB images. Training specialized multi-spectral models is expensive and produces narrow, task-specific systems.

The proposed technique works within the inference pipeline of existing LMMs, like Gemini 2.5, without any additional training. It first adapts non-RGB inputs (e.g., near-infrared or shortwave infrared bands) into the visual space the LMM already understands. Then, it injects domain-specific instructions and chain-of-thought reasoning prompts to guide the model's analysis. The researchers demonstrated strong zero-shot performance gains on popular remote sensing benchmarks, showing that geospatial professionals can now leverage powerful generalist models for specialized sensor inputs, benefiting from rich reasoning capabilities grounded in multi-spectral data.