Structural Regression Fusion for Unsupervised Multimodal Change Detection

Multimodal change detection (MCD) is an increasingly interesting but very challenging topic in remote sensing, which is due to the unavailability of detecting changes by directly comparing multimodal images from different domains. In this paper, we first analyze the structural asymmetry between multitemporal images and show their negative impact on the previous MCD methods using image structures. Specifically, when there is a structural asymmetry, previous structure based methods can only complete a structure comparison or image regression in one direction and fails in the other direction, that is, they cannot transform or convert from complex structural images (with more categories) to simple structural images (with fewer categories). To reduce the influence of structural asymmetry, we propose a structural regression fusion based method (SRF) that simultaneously transforms the pre-event and post-event images into the image domain of each other, calculating the forward and backward changed images, respectively. Noteworthy, different from previous late fusion methods that fuse the forward and backward changed images in the post-processing stage, SRF incorporates fusion into the regression process, which can fully explore the connection between changed images, and thus improve image transformation performance and obtain better changed images. Specifically, SRF yields three types of constraints to perform the fused image transformation: structure consistency based regression term, change smoothness and alignment based fusion term, and prior sparsity based penalty term. Finally, the changes can be extracted by comparing the transformed and original images. The proposed SRF is verified on six real data sets by comparing with some state-of-the-art methods. Source code of the proposed method will be made available at https://github.com/yulisun/SRF.