Two Spectral–Spatial Implicit Neural Representations for Arbitrary-Resolution Hyperspectral Pansharpening

Standard hyperspectral (HS) pansharpening utilizes panchromatic (PAN) images to improve the connected low-resolution HS (LRHS) images to the spatial resolutions of PANs; while arbitrary-resolution hyperspectral (ARHS) pansharpening aims to use PANs to enhance LRHS images to any desired spatial resolutions. For the challenging task of ARHS pansharpening, one of major obstacles is how to generalize the single pansharpening model learned under predetermined training scales to any pansharpening scales for future data. As implicit neural representations (INRs) have a potential to approximate continuous functions, they offer a possible alternative way to naturally resolve ARHS pansharpening. In this paper, we develop two spectral-spatial INRs for ARHS pansharpening: one is a naive pansharpening INR (NaivePINR); the other is a dynamic pansharpening INR (DynamicPINR). The former builds a novel spectral-spatial encoding to produce spectral-spatial priors of observed scenes and uses a spectral-spatial query mapping to reconstruct fine spectral details and spatial details. The latter establishes a innovative two-fold tuning mechanism to dynamically adjust both the spectral-spatial encoding and the spectral-spatial query mapping. Experimental results on serval datasets verify the excellent performances of the proposed pansharpening INRs.