Fusing Multiple Deep Models for In Vivo Human Brain Hyperspectral Image Classification to Identify Glioblastoma Tumor

Glioblastoma (GBM) tumor is the most common primary brain malignant tumor. The precise identification of GBM tumor is very important for diagnosis and treatment. Hyperspectral imaging is a fast, non-contact, accurate and safety modern medical detection technology, which is expected to be a new tool of intraoperative diagnosis. In order to make full use of the spectral and spatial information of hyperspectral images (HSIs) to achieve accurate GBM tumor identification, a method based on fusion of multiple deep models (FMDM) is proposed for in-vivo human brain HSI classification. The proposed method includes the following major steps: (1) spectral phasor analysis and data over-sampling; (2) one-dimensional deep neural network (1D-DNN) based spectral hyperspectral image feature extraction and classification; (3) two-dimensional convolution neural network (2D-CNN) based spectral-spatial hyperspectral image feature extraction and classification; (4) edge-preserving filtering based classification result fusion and optimization; (5) fully convolutional network (FCN) based background segmentation. To verify the capabilities of the proposed method, experiments are performed on two real human brain hyperspectral data sets including 36 in-vivo hyperspectral images captured from 16 different patients. The proposed method can achieve an overall accuracy of 96.69% for four-class classification, and an overall accuracy of 96.34% for GBM tumor identification. Experimental results demonstrate that the proposed method exhibits competitive classification performance and can generate satisfactory thematic maps of the location of the GBM tumor, which can provide the surgeon with guidance on successful and precise tumor resection.