Double Attention Transformer for Hyperspectral Image Classification

CNN has become one of the most popular tools to tackle HSI classification tasks. However, CNN suffers from the long-range dependencies problem, which may degrade the classification performance. To address this issue, this paper proposes a transformer-based backbone network for HSI classification. The core component is a newly designed double-attention transformer encoder (DATE) which contains two self-attention modules, termed as spectral attention module (SPE) and spatial attention module (SPA). SPE extracts the global dependency among spectral bands, and SPA mines the local features of spatial correlation information among pixels. The local spatial tokens and the global spectral token are fused together and updated by SPA. In this way, DATE can not only capture the global dependence among spectral bands, but also extract the local spatial information, which greatly improves the classification performance. To reduce the possible information loss as the network depth increases, a new skip connection mechanism is devised for cross-layer feature fusion. Experimental results in several datasets indicate that the new algorithm holds very competitive classification performance compared to the state-of-the-art methods.