Learning SAR-to-Optical Image Translation via Diffusion Models With Color Memory

Synthetic aperture radar (SAR) and optical sensing are two important means of Earth observation. SAR-to-optical image translation (S2OIT) can integrate the advantages of both and assist SAR image interpretation under all-day and all-weather conditions. The existing S2OIT methods generally follow the generative adversarial networks paradigm, and encounter the problem of mode collapse, making them difficult to train. SAR and optical images have heterogeneous characteristics and large spectral differences, most of the existing methods do not focus on the color correlation between these two image domains, which leads to spectral distortion and detail errors in translation results. To address these issues, we propose a novel diffusion model capable of memorizing color and directly mapping between the SAR and optical image domains for S2OIT called CM-Diffusion. The color attention Brownian bridge diffusion structure is designed to learn the color correlation and translation between the SAR and optical image domains directly through the bidirectional diffusion process and color attention mechanism, avoiding the conditional information leverage. The color feature extraction module is constructed to provide color and semantic information for the diffusion model. Extensive experiments conducted on three benchmark datasets SEN1-2, QXS-SAROPT, and SEN12MS demonstrate that the proposed CM-Diffusion outperforms the state-of-the-art methods on both subjective and objective evaluation metrics.