AtmoFuseNet fuses cameras and radar for 4D cloud reconstruction

Dense volumetric reconstruction of cloud microphysical fields from sparse ground-based instruments has been a persistent challenge due to heterogeneous modalities and uneven spatial coverage. Xinze Zhang presents AtmoFuseNet, a framework that addresses this by fusing multi-view sky camera imagery with millimeter-wave cloud radar and ceilometer observations. The system first uses a cross-modal hierarchical aggregation module that combines image feature pyramids with instrument-derived vertical profiles via layer-wise cross-attention. Next, a conditional variational refinement module maps the resulting volume into physically consistent microphysical fields, guided by differentiable radar and image forward models. Finally, a correlation-based motion estimator recovers per-voxel 3D wind vectors from consecutive volumetric reconstructions.

On collocated observations from a semi-arid site, AtmoFuseNet achieves 0.026 g/m³ liquid water content mean absolute error (MAE) and 1.18 m/s wind speed MAE—significant improvements over existing retrieval baselines. Ablation experiments isolate the contribution of each module, confirming the value of cross-modal fusion and physical constraints. This work opens the door to dense, real-time cloud profiling from ground sensors, with implications for weather forecasting, climate modeling, and aviation safety. The arXiv preprint (2606.30647) provides full architectural details and code references.