Deep Learning With an Attention Mechanism for Differentiating the Origin of Brain Metastasis Using MR images

BACKGROUND: Brain metastasis (BM) is a serious neurological complication of cancer of different origins. The value of deep learning (DL) to identify multiple types of primary origins remains unclear. PURPOSE: To distinguish primary site of BM and identify the best DL models. STUDY TYPE: Retrospective. POPULATION: A total of 449 BM derived from 214 patients (49.5% for female, mean age 58 years) (100 from small cell lung cancer [SCLC], 125 from non-small cell lung cancer [NSCLC], 116 from breast cancer [BC], and 108 from gastrointestinal cancer [GIC]) were included. FIELD STRENGTH/SEQUENCE: A 3-T, T1 turbo spin echo (T1-TSE), T2-TSE, T2FLAIR-TSE, DWI echo-planar imaging (DWI-EPI) and contrast-enhanced T1-TSE (CE T1-TSE). ASSESSMENT: Lesions were divided into training (n = 285, 153 patients), testing (n = 122, 93 patients), and independent testing cohorts (n = 42, 34 patients). Three-dimensional residual network (3D-ResNet), named 3D ResNet6 and 3D ResNet 18, was proposed for identifying the four origins based on single MRI and combined MRI (T1WI + T2-FLAIR + DWI, CE-T1WI + DWI, CE-T1WI + T2WI + DWI). DL model was used to distinguish lung cancer from non-lung cancer; then SCLC vs. NSCLC for lung cancer classification and BC vs. GIC for non-lung cancer classification was performed. A subjective visual analysis was implemented and compared with DL models. Gradient-weighted class activation mapping (Grad-CAM) was used to visualize the model by heatmaps. STATISTICAL TESTS: The area under the receiver operating characteristics curve (AUC) assess each classification performance. RESULTS: 3D ResNet18 with Grad-CAM and AIC showed better performance than 3DResNet6, 3DResNet18 and the radiologist for distinguishing lung cancer from non-lung cancer, SCLC from NSCLC, and BC from GIC. For single MRI sequence, T1WI, DWI, and CE-T1WI performed best for lung cancer vs. non-lung cancer, SCLC vs. NSCLC, and BC vs. GIC classifications. The AUC ranged from 0.675 to 0.876 and from 0.684 to 0.800 regarding the testing and independent testing datasets, respectively. For combined MRI sequences, the combination of CE-T1WI + T2WI + DWI performed better for BC vs. GIC (AUCs of 0.788 and 0.848 on testing and independent testing datasets, respectively), while the combined MRI approach (T1WI + T2-FLAIR + DWI, CE-T1WI + DWI) could not achieve higher AUCs for lung cancer vs. non-lung cancer, SCLC vs. NSCLC. Grad-CAM helped for model visualization by heatmaps that focused on tumor regions. DATA CONCLUSION: DL models may help to distinguish the origins of BM based on MRI data. EVIDENCE LEVEL: 3 TECHNICAL EFFICACY: Stage 2.