Multispectral Image Pan-Sharpening Guided by Component Substitution Model

Multispectral image pan-sharpening aims to increase the spatial details of multispectral images by fusing multispectral and panchromatic images. Existing component substitution-based deep learning pan-sharpening is generally regarded as a black box and fails to mine the image interaction relation with physical significance in each step of pan-sharpening, which not only limits the improvement of image resolution, but also ignores the physical interpretability of the models. To improve this situation, according to the traditional component substitution-based detail injection pan-sharpening model, we consider the matrix calculation in each step as the transformation between image pixel values and carry out linear transformations, and therefore the pan-sharpened multispectral image is represented as the sum of two multispectral images. Then given the spatial and spectral heterogeneity, the two summed images are decomposed based on the fact that any real number can be expressed as the product of two real numbers. Ultimately, the multispectral image pan-sharpening model can be constructed as the sum of two Hadamard products. We design a dual-branch network with attention mechanisms that merges the sum and the Hadamard products into a concise formulation. This method not only enhances physical interpretability but also improves spatial resolution. Experiments on five real-world datasets validate that the proposed multispectral image pan-sharpening model can improve performance.