DCRF-Net: Dual-Observation Consistency Residual Feedback Network for Hyperspectral and Multispectral Image Fusion

Hyperspectral and multispectral image (HSI–MSI) fusion aims to recover a high-spatial-resolution hyperspectral image (HR-HSI) from a low-spatial-resolution hyperspectral image (LR-HSI) and a high-spatial-resolution multispectral image (HR-MSI). The two observations provide complementary spatial and spectral information, and their relationships to the target image are described by spatial and spectral observation models. Many existing deep fusion methods exploit these models through training losses or model-inspired network designs, but the residuals between the current reconstruction and the two observations are less explicitly reused as inference-time feedback. To reuse these residuals during inference, we present DCRF-Net, a dual-observation consistency residual feedback network for standard nonblind HSI–MSI fusion with fixed nominal spatial and spectral operators. Starting from an initial fused estimate, DCRF-Net projects the current HR-HSI into the HR-MSI and LR-HSI observation domains, computes the corresponding residuals, lifts them into the HR-HSI space, and uses a shared feedback cell to predict bounded updates. The method does not estimate degradation parameters during inference. Experiments on PaviaU, Chikusei, and Houston18, together with controlled ablations, show consistent reconstruction gains under a unified matched-degradation protocol.