Spatial–Spectral Attention Pyramid Network for Hyperspectral Stripe Restoration

Push-broom hyperspectral imaging systems often suffer from stripe artifacts. The conventional methods treat the artifacts as noise and suppress narrow-stripe ones well, but show limitations to wide and full-band stripe artifacts. To address the problem, this paper proposes a spatial-spectral attention pyramid network (SAPN) for hyperspectral stripe restoration. Firstly, the spatial-spectral mixed attention module (SMA) is developed to tackle the inefficiency of neighborhood representation in wide stripes, and it is mainly composed of three spatial and spectral attention (SSA) operations. Each SSA specifically combines channel and non-local attention to compute spatial-spectral attention features. SMA utilizes these SSA operations to achieve different spatial-spectral attention features for multi-directional slices of hyperspectral cubes, and then fuses them to establish the contextual connection between the single pixel and the global information. Further, we build an efficient pyramid backbone (EPB) for stripe restoration. In EPB, multi-resolution shallow pyramid features are extracted by the lightweight head module, and then inferred and reconstructed by SMA and other layers from coarse to fine, the shareable SSA layer also greatly decreases parameters. Besides, we develop an unsupervised learning strategy where SAPN generates pseudo-reference images with the aid of deep image prior, and achieve the convergent model for batch images. Experiments are carried out on the private and public hyperspectral datasets where wide stripes respectively exist at the same and different spatial locations in all bands. Experimental results demonstrate that SAPN can obtain competitive objective metrics, and it can restore images with more realistic texture and fidelity spectra.