TransPND: A Transformer Based Pulmonary Nodule Diagnosis Method on CT Image

Detection of Benign and malignant pulmonary nodules is a significant help for early lung cancer diagnosis. Owing to the superior performance of the transformer based deep learning methods in different computer vision tasks, this study attempts to introduce it into the CT image classification task of pulmonary nodules. However, the problems of rare samples and harrowing local feature extraction in this field still need to solve. To this end, we introduce a CT image-based transformer for pulmonary nodule diagnosis (TransPND). Specifically, firstly, we introduce a 2D Panning Sliding Window (2DPSW) for data enhancement, making it more focused on local features. Secondly, unlike the encoder of the traditional transformer, we divide the encoder part of TransPND into two parts: Self Attention Encoder (SA) and Directive Class Attention Encoder (DCA). SA is similar to the traditional self-attention mechanism, except that we introduce Local Diagonal Masking (LDM) to select the attention location and focus on the correlation between tokens rather than itself score. Meanwhile, based on SA, we improve it and propose DCA to guide attention to focus more on local features and reduce computational effort. Finally, to solve the model overfitting problem caused by the increasing depth, we choose the Weight Learning Diagonal Matrix (WLDM) to gate each residual connection in both the SA and DCA stages. We conducted extensive experiments on the LIDC-IDRI dataset. The experimental results show that our model achieves an accuracy of 93.33 $$\%$$ compared to other studies using this dataset for lung nodule classification. To the best of our knowledge, TransPND is the first research on the classification of lung nodule CT images based on pure transformer architecture.