Salient Difference Awareness for Remote Sensing Image Change Captioning

Remote sensing image change captioning is an innovative human-computer interaction for understanding appeared changes in bi-temporal remote sensing images (RSIs), which provides a natural language description for changes or non-changes. Current methods typically adopt a bi-temporal feature extraction, change localization and followed by a change captioning model. Most approaches aim to improve the accuracy of change recognition and description with complex network modules, lacking prior judgments on whether there are salient changes between image pairs. This deficiency can lead to inaccurate or erroneous descriptions. To alleviate this problem, a novel salient difference awareness network is designed to describe the differences between RSI pairs. Generally, the change area accounts for a relatively small proportion in a RSI, the salient changeaware module filters the redundant information in RSI scene, which benefits change localization by attaining better change perception. In addition, we propose multilayer aggregated module to sufficiently utilize the salient change features from different change awareness layers, further capturing relevant change details. Then, the Transformer decoder guarantees semantic descriptions by change-to-sentence translation. We validate the effectiveness of the proposed method on the LEVIR-CC dataset with four metrics, showcasing its superior performance compared to several state-of-the-art techniques.