A Self-Attention Based Fusion Model of Radiomics and Deep Features for Early Recurrence Prediction in NSCLC

Non-small cell lung cancer (NSCLC) is a prevalent malignant tumor with high mortality and recurrence rates. Accurate prediction of early recurrence is crucial for guiding early-stage treatment and improving survival rates. This paper proposed a deep learning model based on self-attention mechanism for NSCLC early recurrence prediction. Initially, we employ diverse machine learning techniques to extract handcrafted features from CT images, encompassing texture, shape, grayscale, etc. Subsequently, a pre-trained ResNet50 network is utilized to extract deep features that encapsulate high-level semantic and representation information from the CT images. These features are then fused into a unified feature vector. To enhance prediction accuracy and robustness, an innovative self-attention fusion module is designed. Leveraging the self-attention mechanism, this module optimizes and weights the fused feature vector effectively. Experimental results on the public Cancer Imaging Archive (TCIA) dataset demonstrate that our model outperforms existing methods in early recurrence prediction, exhibiting significant improvements in classification accuracy, sensitivity, specificity, and AUC.