HIMEC AI model tops remote sensing change captioning with CIDEr 142.8

Remote sensing image change captioning (RSICC) aims to automatically describe semantic changes between two satellite images taken at different times. Most existing methods directly condition a caption decoder on fused visual features, leaving the intermediate change structure and decoder-interface consistency unexplored. The new arXiv paper introduces HIMEC, a method that combines Directional Change Representation (DCR) with fixed-interface decoding. DCR separates signed differences into appearance-oriented, disappearance-oriented, and shared-context streams before fusion. A learned-query encoder then converts the fused representation into visually conditioned change-query tokens, which serve as the scene decoder's only sample-dependent memory. During training, an auxiliary phrase decoder provides caption-derived supervision. With a fixed zero input, the scene decoder maintains the same interface during training and inference, ensuring consistency.

HIMEC's performance is validated on two benchmarks. On the changed LEVIR-CC validation pairs, HIMEC achieves a CIDEr score of 142.81 ± 0.60, compared to 139.51 ± 3.40 for a direct fused-feature memory baseline. On SECOND-CC, fixed-zero and regime-matched diagnostic conditioning reach 75.67 and 76.99 CIDEr, respectively, versus 60.77 for a mismatched cascade. The authors also evaluate a local-to-scene cascade conditioned on teacher-forced local states during training and autoregressive states at inference, showing a mean cosine distance of 0.69 between these state types; regime-matched conditioning recovers most of the associated deficit. The paper is submitted to IEEE Transactions on Geoscience and Remote Sensing (TGRS), and the source code will be made publicly available upon publication.