Deep Learning for Lung Cancer Nodal Staging and Real-World Clinical Practice

HomeRadiologyVol. 302, No. 1 PreviousNext Reviews and CommentaryFree AccessEditorialDeep Learning for Lung Cancer Nodal Staging and Real-World Clinical PracticeChang Min Park , Jong Hyuk LeeChang Min Park , Jong Hyuk LeeAuthor AffiliationsFrom the Department of Radiology, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea; and Department of Radiology, Seoul National University Hospital, Seoul, Korea.Address correspondence to C.M.P. (e-mail: [email protected]).Chang Min Park Jong Hyuk LeePublished Online:Oct 26 2021https://doi.org/10.1148/radiol.2021211981MoreSectionsPDF ToolsImage ViewerAdd to favoritesCiteTrack CitationsPermissionsReprints ShareShare onFacebookTwitterLinked In See also the article by Zhong and She et al in this issue.Dr Chang Min Park is an associate professor in the Department of Radiology at Seoul National University Hospital. His research interests focus on artificial intelligence technology for radiologic imaging (development, validation, and clinical application), imaging analysis of lung cancers/nodules, and thoracic interventions. His research goal is to improve diagnostic accuracy and the health outcomes of patients with disease and, ultimately, to enhance the health of people worldwide through innovative technology.Download as PowerPointOpen in Image Viewer Dr Jong Hyuk Lee is an assistant professor in the thoracic imaging section of the Department of Radiology at the Seoul National University Hospital. His research interests focus on the validation and clinical application of artificial intelligence technology, diagnosis of various thoracic diseases, and thoracic interventions.Download as PowerPointOpen in Image Viewer Lung cancer staging is essential for planning the treatment strategy, predicting the prognosis, and evaluating treatment results (1). For lung cancer with a nodal category of N2 or higher, multidisciplinary treatment combining surgery and chemoradiation therapy is offered due to its survival benefits (2). CT and PET are routinely performed to correctly diagnose N2 disease before treatment; however, their sensitivity for N2 disease ranges from only 70% to 85% (1). Endobronchial US with transbronchial needle aspiration can more accurately assess the nodal category of lung cancer (1). However, its diagnostic value in clinical N0 lung cancer remains unclear (3). Indeed, up to 8.5% of clinical N0 lung cancers were found to have N2 metastasis at pathologic examination (4).In this issue of Radiology, Zhong and She et al addressed this issue of prediction of N2 metastasis using a deep learning technique (5). They developed a deep learning–based prediction model for N2 disease in clinical stage I non–small cell lung cancers (NSCLCs) using chest CT images. The authors validated the model through multiple sessions of testing for the prediction of N2 disease and compared the model’s performance with that of three clinical models (the Veterans Affairs model, Fudan model, and Beijing model). As a result, the deep learning model demonstrated areas under the receiver operating characteristic curve (AUCs) of 0.81–0.82, which were significantly higher than those obtained using the three clinical models. In addition, the AUC of the model was higher than that with maximum standardized uptake value on PET (AUC: 0.57) in the prospective cohort test set.Subsequently, Zhong and She et al suggested the biologic basis for this deep learning model by verifying the association of the model’s risk score with gene expression patterns (eg, EGFR or ALK mutations). Furthermore, they demonstrated that the model’s risk scores also stratified patients’ overall survival and recurrence-free survival. In the Cox regression analyses, the model’s risk score was a significant prognostic factor for overall survival and recurrence-free survival along with other factors including age, sex, nodule type (subsolid nodule), and pathologic nodal stage. Finally, the model predicted the benefits of adjuvant chemotherapy in patients with moderate-to-high risk scores.This study dealt with a clinically relevant topic that has not been satisfactorily addressed to date. Lung cancer with N2 lymph node metastasis is at least stage III and necessitates a multidisciplinary treatment approach (1). Therefore, accurate nodal categorization in the initial staging work-up is critical for establishing an appropriate treatment strategy. Nevertheless, lung cancer with pathologic N2 nodal involvement is still incorrectly diagnosed as N0 or N1 lung cancer during the initial clinical staging in up to 8.5% of patients with clinical stage I NSCLC (4). This limitation might hamper radiologists and clinicians from establishing the most appropriate treatment strategy for patients, leading to less favorable outcomes (1). Because the model’s performance was found to be better than those of the tested clinical models and other currently used techniques (eg, PET) for predicting N2 disease, the authors successfully proposed a new solution with a deep learning technique for this clinically relevant but unresolved issue. With these promising results, this study serves as a bridgehead narrowing the gap between the deep learning field and real-world medical applications.In the field of artificial intelligence and deep learning, the process of model development and verification of a model’s performance is composed of sequential steps of training, validation, and testing (6). To guarantee the generalizability and applicability of a model in clinical practice, sufficient testing steps with external data sets that are independent of the model development and that reflect daily clinical practice are indispensable (7). In accordance with this need, Zhong and She et al conducted multiple testing for their N2 signature model, including an internal test set, an external test set using open-source data, and another external test set of prospectively collected CT data sets from four institutions. In general, the simple development of a deep learning algorithm along with the results of internal testing alone is not sufficient to guarantee a model’s generalizability in real-world clinical settings, which can be different from model development data sets (6). Instead, based on the results of multiple sessions of thorough tests, the authors could make the robust conclusion that their N2 signature model can be generalized in real-world clinical practice.The composition of the test data set is another critical point. A diagnostic case-control study in which researchers collect disease-positive and disease-negative cases through convenience sampling cannot reflect the real-world clinical practices. Diagnostic case-control studies of this type often have unrealistic disease prevalence or disease spectrum and are prone to overestimating the model’s diagnostic performance (7). In contrast, the test data sets of this study were composed of diagnostic cohorts including all patients with clinical stage I NSCLC between January 2011 and December 2013 and those with surgically resected NSCLCs between May 2020 and October 2020 as a multicenter prospective test cohort. The prevalence of N2 nodal involvement in the three test data sets ranged between 10% and 10.7%, reflecting the real-world prevalence of N2 disease among clinical N0 lung cancers (4). These points, in our opinion, increase the probability that this model can work well in routine practice.Another strength of this study is that Zhong and She et al compared the model’s performance with three clinical models for predicting N2 nodal involvement. These clinical models used various pathologic (pathologic type and tumor size), radiologic (whether patients have abnormal chest radiographs, tumor location), and clinical (primary symptom, age) information. In contrast, the study’s signature model used only information from CT images and, interestingly, showed higher predictive performance for N2 disease. Indeed, clinicians have claimed that it is difficult to adopt certain types of clinical models because they require complex information from various sources, and the lack of this information hampers the applicability of these clinical models (8). Thanks to the simplicity of using only CT images, it is expected that the clinical applicability of this model would be better than that of conventional models.It is noteworthy that Zhong and She et al investigated the association between the model and radiogenomics through a gene alteration analysis and gene set enrichment analysis to explore its biologic basis. A recent study successfully applied a deep learning technique in genetics to investigate novel aspects of gene expression, such as the genetic regulatory code that controls messenger RNA abundance (9). Likewise, the study by Zhong and She et al demonstrated an association between various gene expression patterns and the results of deep learning algorithms in patients with NSCLC (5,9). In addition, the authors demonstrated the prognostic implications of the model signature in the internal and external test sets. That is, overall survival and recurrence-free survival were stratified according to the score of the model signature in all patients and in patients receiving postoperative chemotherapy. These results regarding gene expression and survival prediction may help explain the black-box characteristics (ie, an unknowability of how a particular prediction is achieved), which is one of the well-known drawbacks of deep learning techniques (10). Nevertheless, it is still unclear whether the association of the model with gene expression is a simple reflection of the model output or the original gene-related basis of the model, for which further research is warranted.This study serves as an excellent example of deep learning research with the selection of clinically relevant topics, build-up of a simply applicable model structure, robust tests reflecting real-world clinical practice, and exploration of the biologic basis of the observed associations. Through high-quality deep learning studies satisfying these criteria, deep learning will be successfully implemented in real-world clinical practice in the near future.Disclosures of Conflicts of Interest: C.M.P. received grants from Lunit; participates on a data safety monitoring board or advisory board at Seoul National University Hospital; has a leadership or fiduciary role in other board, society, committee, or advocacy group, paid or unpaid, from Korean Society of Radiology, Korean Society of Thoracic Radiology, and Korean Society of Artificial Intelligence in Medicine; has stock in Promedius and stock options in Lunit and Coreline Soft. J.H.L. disclosed no relevant relationships.References1. Rami-Porta R, Call S, Dooms C, et al. Lung cancer staging: a concise update. Eur Respir J 2018;51(5):1800190. Crossref, Medline, Google Scholar2. Scagliotti GV, Pastorino U, Vansteenkiste JF, et al. Randomized phase III study of surgery alone or surgery plus preoperative cisplatin and gemcitabine in stages IB to IIIA non–small-cell lung cancer. J Clin Oncol 2012;30(2):172–178. Crossref, Medline, Google Scholar3. Vial MR, O’Connell OJ, Grosu HB, et al. Diagnostic performance of endobronchial ultrasound-guided mediastinal lymph node sampling in early stage non–small cell lung cancer: a prospective study. Respirology 2018;23(1):76–81. Crossref, Medline, Google Scholar4. Lin JT, Yang XN, Zhong WZ, et al. Association of maximum standardized uptake value with occult mediastinal lymph node metastases in cN0 non-small cell lung cancer. Eur J Cardiothorac Surg 2016;50(5):914–919. Crossref, Medline, Google Scholar5. Zhong Y, She Y, Deng J, et al. Deep learning for prediction of N2 metastasis and survival for clinical stage I non–small cell lung cancer. Radiology 2021.https://doi.org/10.1148/radiol.2021210902. Published online October 19, 2021. Link, Google Scholar6. Park SH, Han K. Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology 2018;286(3):800–809. Link, Google Scholar7. Park SH. Diagnostic case-control versus diagnostic cohort studies for clinical validation of artificial intelligence algorithm performance. Radiology 2019;290(1):272–273. Link, Google Scholar8. Lu MT, Raghu VK, Mayrhofer T, Aerts HJWL, Hoffmann U. Deep learning using chest radiographs to identify high-risk smokers for lung cancer screening computed tomography: development and validation of a prediction model. Ann Intern Med 2020;173(9):704–713. Crossref, Medline, Google Scholar9. Zrimec J, Börlin CS, Buric F, et al. Deep learning suggests that gene expression is encoded in all parts of a co-evolving interacting gene regulatory structure. Nat Commun 2020;11(1):6141. Crossref, Medline, Google Scholar10. Castelvecchi D. Can we open the black box of AI? Nature 2016;538(7623):20–23. Crossref, Medline, Google ScholarArticle HistoryReceived: Aug 4 2021Revision requested: Aug 25 2021Revision received: Aug 25 2021Accepted: Aug 27 2021Published online: Oct 26 2021Published in print: Jan 2022 FiguresReferencesRelatedDetailsAccompanying This ArticleDeep Learning for Prediction of N2 Metastasis and Survival for Clinical Stage I Non–Small Cell Lung CancerOct 26 2021RadiologyRecommended Articles Deep Learning for Prediction of N2 Metastasis and Survival for Clinical Stage I Non–Small Cell Lung CancerRadiology2021Volume: 302Issue: 1pp. 200-211Recurrence Patterns and Patient Outcomes in Resected Lung Adenocarcinoma Differ according to Ground-Glass Opacity at CTRadiology2023Volume: 307Issue: 3Deep Learning Demonstrates Potential for Lung Cancer Detection in Chest RadiographyRadiology2020Volume: 297Issue: 3pp. 697-698Mediastinal Lymphadenopathy in Lung Cancer Screening: A Red FlagRadiology2021Volume: 302Issue: 3pp. 695-696Subsolid Lung Nodules: Potential for OverdiagnosisRadiology2019Volume: 293Issue: 2pp. 449-450See More RSNA Education Exhibits Role Of Radiology In Addressing The Challenge Of Lung Cancer After Lung Transplantation.Digital Posters2021Multifocal Lung Adenocarcinomas: Comprehensive Review and the Current Management StrategyDigital Posters2019Introduction to Artificial Intelligence and Big Data Research in Chest RadiologyDigital Posters2019 RSNA Case Collection Diffuse idiopathic pulmonary neuroendocrine cell hyperplasiaRSNA Case Collection2020Thoracic splenosisRSNA Case Collection2020Lipoid PneumoniaRSNA Case Collection2021 Vol. 302, No. 1 Metrics Altmetric Score PDF download