HybridSN: Exploring 3-D–2-D CNN Feature Hierarchy for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is widely used for the analysis of\nremotely sensed images. Hyperspectral imagery includes varying bands of images.\nConvolutional Neural Network (CNN) is one of the most frequently used deep\nlearning based methods for visual data processing. The use of CNN for HSI\nclassification is also visible in recent works. These approaches are mostly\nbased on 2D CNN. Whereas, the HSI classification performance is highly\ndependent on both spatial and spectral information. Very few methods have\nutilized the 3D CNN because of increased computational complexity. This letter\nproposes a Hybrid Spectral Convolutional Neural Network (HybridSN) for HSI\nclassification. Basically, the HybridSN is a spectral-spatial 3D-CNN followed\nby spatial 2D-CNN. The 3D-CNN facilitates the joint spatial-spectral feature\nrepresentation from a stack of spectral bands. The 2D-CNN on top of the 3D-CNN\nfurther learns more abstract level spatial representation. Moreover, the use of\nhybrid CNNs reduces the complexity of the model compared to 3D-CNN alone. To\ntest the performance of this hybrid approach, very rigorous HSI classification\nexperiments are performed over Indian Pines, Pavia University and Salinas Scene\nremote sensing datasets. The results are compared with the state-of-the-art\nhand-crafted as well as end-to-end deep learning based methods. A very\nsatisfactory performance is obtained using the proposed HybridSN for HSI\nclassification. The source code can be found at\n\\url{https://github.com/gokriznastic/HybridSN}.\n