M3R: Localized Rainfall Nowcasting with Meteorology-Informed MultiModal Attention

A research team led by Sanjeev Panta has introduced M3R (Meteorology-informed MultiModal attention-based architecture for direct Rainfall prediction), a novel AI system that significantly advances short-term precipitation forecasting. The architecture addresses a critical limitation in current deep learning approaches by effectively leveraging diverse multimedia data sources through a comprehensive pipeline for temporal alignment of heterogeneous meteorological data. M3R combines visual NEXRAD radar imagery with numerical Personal Weather Station (PWS) measurements, using specialized multimodal attention mechanisms that treat weather station time series as queries to selectively attend to spatial radar features.

This innovative approach enables focused extraction of precipitation signatures that traditional methods often miss. Experimental results demonstrate that M3R outperforms existing approaches for three spatial areas of 100 km × 100 km centered at NEXRAD radar stations, achieving substantial improvements in accuracy, efficiency, and precipitation detection capabilities. The model establishes new benchmarks for multimedia-based precipitation nowcasting and provides practical tools that could be integrated into operational weather prediction systems. The research, accepted at IEEE International Conference on Multimedia and Expo (ICME) 2026, represents a significant step forward in disaster mitigation and water resource management through more precise localized forecasting.