Unsupervised Blind Spectral-spatial Cross-super-resolution Network for HSI and MSI Fusion

A high-spatial-resolution hyperspectral image (HR-HSI) can be obtained by fusing a hyperspectral image and a multispectral image (HSI-MSI) since it takes the advantage of combing the low-spatial-resolution hyperspectral image (LR-HSI) and the high-spatial-resolution multispectral image (HR-MSI). Currently, most HSI-MSI fusion methods decompose HSIs in both spatial and spectral domains, which destroys the correlation of the two domains and results in poor fusion results. To further explore the correlation of spatial and spectral information, an unsupervised blind spectral-spatial cross-super-resolution network (CSRNet) is proposed for HSI-MSI fusion. The network transforms the fusion into two super-resolution reconstructions for a HR-HSI to avoid the decomposition of HSIs and achieves interaction with the two reconstructions to provide strong constraints for each other. The network uses the following procedure. First, the HSI-MSI fusion is transformed into a spectral-spatial cross-super-resolution model, which consists two branches: one for spatial and the other for spectral super-resolution reconstructions. As the basic blocks of the two branches, spatial super-resolution (SpaSR) and the spectral super-resolution (SpeSR) blocks in the new network are designed via iterations unfolding from half quadratic splitting (HQS). Then their corresponding spatial constraint (SpaC) and spectral constraint (SpeC) modules are established for the mutual constraints and interactions between the two branches. The SpaSR/SpeSR and SpaC/SpeC modules are alternately connected to form the above spatial/spectral super-resolution branches for reconstructing a HR-HSI. Both visual and quantitative results of experiments based on both simulated and real observation datasets showed that the proposed method outperformed seven commonly used methods, suggesting that the new method is effective for preserving the correlation of spatial and spectral information in HR-HSIs.