Unsupervised Blur Kernel Learning for Pansharpening

Deep learning (DL) for pansharpening has recently attracted considerable attentions. To construct training data, DL based pansharpening approaches often downsample the original multispectral image (MSI) and panchromatic image (PAN) with fixed blur kernel, which can be different from the real point spread functions (PSF) of the satellites. And a mismatched blur kernel will cause the pansharpening performance to drop dramatically. In this paper, we propose a novel blur kernel learning method for pansharpening, which can learn the spatial and spectral blur kernels between PAN and MSI in an unsupervised way. Specifically, we analyze the relationship between PAN and MSI, and then construct a mini net for blur kernel learning. Once the spatial blur kernel is found, a convolutional neural network (CNN) for pansharpening is trained on the downsampled dataset using the learned spatial blur kernel. Experimental results on GF-2 images demonstrate the superiority of the proposed method.