Dual-Stream Discriminative Attention Network for Cross-Scene Hyperspectral Image Classification

In hyperspectral image (HSI) classification, the challenge of the small-sample-size problem persists as a significant obstacle due to the high cost of labeling samples. To effectively train models with a limited sample set, the application of a transfer learning approach called cross-scene HSI classification is considered a viable solution to address this problem. In cross-scene HSI classification, a source scene with sufficient labeled samples is leveraged to assist in classifying a target scene that lacks labeled samples. Considering that real HSIs may be captured by different sensors, we propose a novel heterogeneous transfer learning algorithm called dual-stream discriminative attention network (DSDAN) to address the task of cross-scene HSI classification. The DSDAN predominantly comprises three pivotal modules. 1) A dual-stream lightweight hybrid CNN (DSLHC) incorporates both the source stream and the target stream is applied to extract alignment spatial-spectral features from heterogeneous data. 2) A discriminative attention block (DAB) is created to address the domain shift between two scenes. Following the DSLHC, the DAB assigns discriminative attention weights to the source features, facilitating a closer alignment of features from two scenes. 3) A specially designed cross-domain loss (CDL) is designed to drive intra-class samples from two scenes to become more consistent, while inter-class samples from two scenes become more distinct, thereby further mitigating domain shift. By combining DSLHC, DAB, and CDL, the complete DSDAN model is established. The effectiveness of DSDAN is validated using three real cross-scene HSI datasets.