Computer-aided diagnosis of ground glass pulmonary nodule by fusing deep learning and radiomics features

Abstract Objectives . This study aims to develop a computer-aided diagnosis (CADx) scheme to classify between benign and malignant ground glass nodules (GGNs), and fuse deep leaning and radiomics imaging features to improve the classification performance. Methods . We first retrospectively collected 513 surgery histopathology confirmed GGNs from two centers. Among these GGNs, 100 were benign and 413 were malignant. All malignant tumors were stage I lung adenocarcinoma. To segment GGNs, we applied a deep convolutional neural network and residual architecture to train and build a 3D U-Net. Then, based on the pre-trained U-Net, we used a transfer learning approach to build a deep neural network (DNN) to classify between benign and malignant GGNs. With the GGN segmentation results generated by 3D U-Net, we also developed a CT radiomics model by adopting a series of image processing techniques, i.e. radiomics feature extraction, feature selection, synthetic minority over-sampling technique, and support vector machine classifier training/testing, etc. Finally, we applied an information fusion method to fuse the prediction scores generated by DNN based CADx model and CT-radiomics based model. To evaluate the proposed model performance, we conducted a comparison experiment by testing on an independent testing dataset. Results . Comparing with DNN model and radiomics model, our fusion model yielded a significant higher area under a receiver operating characteristic curve (AUC) value of 0.73 ± 0.06 ( P < 0.01). The fusion model generated an accuracy of 75.6%, F1 score of 84.6%, weighted average F1 score of 70.3%, and Matthews correlation coefficient of 43.6%, which were higher than the DNN model and radiomics model individually. Conclusions . Our experimental results demonstrated that (1) applying a CADx scheme was feasible to diagnosis of early-stage lung adenocarcinoma, (2) deep image features and radiomics features provided complementary information in classifying benign and malignant GGNs, and (3) it was an effective way to build DNN model with limited dataset by using transfer learning. Thus, to build a robust image analysis based CADx model, one can combine different types of image features to decode the imaging phenotypes of GGN.