Using Deep Learning to Predict Axillary Lymph Node Metastasis from US Images of Breast Cancer

HomeRadiologyVol. 294, No. 1 PreviousNext Reviews and CommentaryFree AccessEditorialUsing Deep Learning to Predict Axillary Lymph Node Metastasis from US Images of Breast CancerMin Sun Bae Min Sun Bae Author AffiliationsFrom the Department of Radiology, Inha University Hospital, 27 Inhang-ro, Jung-gu, Incheon 22332, Korea.Address correspondence to the author (e-mail: [email protected]).Min Sun Bae Published Online:Nov 19 2019https://doi.org/10.1148/radiol.2019192339MoreSectionsPDF ToolsImage ViewerAdd to favoritesCiteTrack CitationsPermissionsReprints ShareShare onFacebookTwitterLinked In See also the article by Zhou et al in this issue.Dr Min Sun Bae is an associate professor of radiology at Inha University Hospital and College of Medicine, Korea. She completed a 2-year research fellowship in breast imaging at Memorial Sloan Kettering Cancer Center. Dr Bae serves on the editorial board of the Korean Journal of Radiology.Download as PowerPointOpen in Image Viewer During the past 2 decades, management of the axilla in early stage breast cancer has evolved. Results from randomized controlled trials have allowed for the de-escalation of axillary surgery in patients with breast cancer (1,2). The American College of Surgeons Oncology Group Z0011 trial showed that axillary lymph node dissection had no survival benefit when there were one or two positive sentinel lymph nodes in patients with clinical T1 or T2 node-negative breast cancer (2). The omission of an invasive axillary procedure prevents morbidity and complications such as hematoma and lymphedema. Randomized controlled trials for omitting sentinel lymph node biopsy are currently under way in Europe. The SOUND (Sentinel Node versus Observation after Axillary Ultrasound) and INSEMA (Intergroup-Sentinel-Mamma) trials investigate whether sentinel lymph node biopsy could be safely omitted in patients with clinically node-negative breast cancer treated with breast conservation therapy (3). Because imaging plays an important role in patient treatment, imaging features predicting lymph node metastasis have the potential to promote more effective decision making.Breast US is a simple and noninvasive method in the preoperative evaluation of patients with breast cancer. US is less expensive than MRI or PET and is more widely used in many countries. It also facilitates easy access when image-guided biopsy is required. Previous studies have shown that US features of invasive primary breast cancer are associated with axillary lymph node metastasis. These US features include a shorter distance of the tumor from the skin and the nipple, larger tumor size, calcifications in the tumor mass, architectural distortion associated with the mass, and increased stiffness at shear-wave elastography (4). However, there is a relatively high interobserver variability in US image acquisition and interpretation.Machine learning is a subset of artificial intelligence in the field of computer science that has been introduced in radiology. Artificial intelligence methods excel at recognizing complex patterns in imaging data and provide a quantitative assessment in an automated manner. Deep learning is a subset of machine learning that is based on artificial neural networks similar to biologic neural networks such as the human brain. The convolutional neural network (CNN) is the most commonly used deep learning algorithm in medical image analysis (5). Deep learning algorithms can automatically learn hierarchical feature representations of the image and can quantify phenotypic characteristics. Given its ability to learn complex image characteristics, deep learning is often robust against the interobserver variability. The most common application of deep learning in breast imaging is detection and characterization of breast lesions at mammography or tomosynthesis (6). In terms of predicting clinical outcomes and prognosis, few studies have used deep learning methods in breast imaging.In this issue of Radiology, Zhou et al (7) demonstrate the potential of deep learning approaches to predict axillary lymph node metastasis based on a US image data set from 834 patients with breast cancer. There were three subgroups of patients: a training and validation set (n = 680), an internal testing set (n = 76), and an external testing set (n = 78). The authors used three different CNN models pretrained on the ImageNet: Inception V3, Inception-ResNet V2, and ResNet 101. The performance of the deep learning models in the prediction of lymph node metastasis was compared with that of one of three radiologists. For the retrospective image analysis, radiologists were trained to learn the US features of breast tumors for predicting axillary lymph node metastasis. Findings at pathologic examination were used as the reference standard. If the three radiologists disagreed on the prediction of lymph node metastasis, another two radiologists interpreted the US images. Consensus radiologist prediction was determined based on the majority of five radiologists. All patients had clinically node-negative T1 or T2 breast cancer and underwent sentinel lymph node biopsy. Among the 834 patients evaluated, 420 had axillary lymph node metastasis. In the majority of patients, the amount of axillary metastatic disease was small: involvement of one or two lymph nodes (n = 329) or only micrometastasis (n = 230).Deep learning models performed well in this study, with the Inception V3 model having the best performance. This model showed an area under the receiver operating characteristic curve of 0.90 and 0.89 in the internal and external test sets, respectively. In the external test set, accuracy of the deep learning models was 79% for Inception V3, 77% for Inception-ResNet V2, and 73% for ResNet 101. The corresponding sensitivities were 85%, 78%, and 73%, respectively, and the corresponding specificities were 73%, 75%, and 73%. Of note, the deep learning models performed better than the radiologists, who had an accuracy of 68%, a sensitivity of 73%, and a specificity of 63% in the external test set.Zhou et al (7) used class activation maps to show which parts of the breast tumor are predictive of axillary lymph node metastasis. Class activation mapping, which provides heat map visualization in the image, can be used to interpret the prediction decision by the neural network. The class activation map helps us understand if the CNN is looking at appropriate parts of the image. At present, we have limited ability to determine the precise logic of deep learning–based predictions.A limitation of the study by Zhou et al (7) is that it was a retrospective study, as the authors point out. The test sets in this study were also relatively small, although an external validation test was performed. In addition, US images were obtained by various physicians and multiple machines. Although this may affect image quality, the results may be more likely to generalize than if a single US machine or reader were used.In conclusion, Zhou et al (7) demonstrate that it is feasible to use deep learning methods to predict axillary lymph node metastasis from US images of breast cancer. The performance of deep learning models was superior to that of radiologists. The deep learning model could be improved if the clinical-pathologic factors are incorporated in modeling to predict axillary lymph node metastasis. Deep learning–based imaging biomarkers may contribute to the advancement of personalized medicine.Disclosures of Conflicts of Interest: disclosed no relevant relationships.References1. Park KU, Caudle A. Management of the axilla in the patient with breast cancer. Surg Clin North Am 2018;98(4):747–760. Crossref, Medline, Google Scholar2. Giuliano AE, Hunt KK, Ballman KV, et al. Axillary dissection vs no axillary dissection in women with invasive breast cancer and sentinel node metastasis: a randomized clinical trial. JAMA 2011;305(6):569–575. Crossref, Medline, Google Scholar3. Reimer T, Hartmann S, Stachs A, Gerber B. Local treatment of the axilla in early breast cancer: concepts from the national surgical adjuvant breast and bowel project B-04 to the planned intergroup sentinel mamma trial. Breast Care (Basel) 2014;9(2):87–95. Crossref, Medline, Google Scholar4. Bae MS, Shin SU, Song SE, Ryu HS, Han W, Moon WK. Association between US features of primary tumor and axillary lymph node metastasis in patients with clinical T1-T2N0 breast cancer. Acta Radiol 2018;59(4):402–408. Crossref, Medline, Google Scholar5. Soffer S, Ben-Cohen A, Shimon O, Amitai MM, Greenspan H, Klang E. Convolutional neural networks for radiologic images: a radiologist's guide. Radiology 2019;290(3):590–606. Link, Google Scholar6. Geras KJ, Mann RM, Moy L. Artificial intelligence for mammography and digital breast tomosynthesis: current concepts and future perspectives. Radiology 2019;293(2):246–259. Link, Google Scholar7. Zhou LQ, Wu XL, Huang SY, et al. Lymph node metastasis prediction from primary breast cancer US images using deep learning. Radiology 2020;294:19–28. Link, Google ScholarArticle HistoryReceived: Oct 18 2019Revision requested: Oct 29 2019Revision received: Oct 31 2019Accepted: Nov 4 2019Published online: Nov 19 2019Published in print: Jan 2020 FiguresReferencesRelatedDetailsCited ByMultimodal radiomics and nomogram‐based prediction of axillary lymph node metastasis in breast cancer: An analysis considering optimal peritumoral regionYayangDuan, XiaoboChen, WanyanLi, SiyaoLi, ChaoxueZhang2023 | Journal of Clinical UltrasoundDeep learning method with a convolutional neural network for image classification of normal and metastatic axillary lymph nodes on breast ultrasonographyJoOzaki, TomoyukiFujioka, EmiYamaga, AtsushiHayashi, YuKujiraoka, TomokiImokawa, KanaeTakahashi, SayuriOkawa, YukaYashima, MioMori, KazunoriKubota, GoshiOda, TsuyoshiNakagawa, UkihideTateishi2022 | Japanese Journal of Radiology, Vol. 40, No. 8From the patient to the population: Use of genomics for population screeningChloeMighton, SalmaShickh, VernieAguda, SuvethaKrishnapillai, EllaAdi-Wauran, YvonneBombard2022 | Frontiers in Genetics, Vol. 13Ultrasound-Based Radiomics Can Classify the Etiology of Cervical Lymphadenopathy: A Multi-Center Retrospective StudyYajingLiu, JifanChen, ChaoZhang, QunyingLi, HangZhou, YiqingZeng, YingZhang, JiaLi, WenXv, WencunLi, JianingZhu, YananZhao, QinChen, YiHuang, HongmingLi, YingHuang, GaoyiYang, PintongHuang2022 | Frontiers in Oncology, Vol. 12Updates in Artificial Intelligence for Breast ImagingManishaBahl2022 | Seminars in Roentgenology, Vol. 57, No. 2Radiomics model based on features of axillary lymphatic nodes to predict axillary lymphatic node metastasis in breast cancerYongTang, XiaolingChe, WeijiaWang, SongSu, YueNie, ChunmeiYang2022 | Medical Physics, Vol. 49, No. 12Toward Generalizability in the Deployment of Artificial Intelligence in Radiology: Role of Computation Stress Testing to Overcome UnderspecificationThomas Eche, Lawrence H. Schwartz, Fatima-Zohra Mokrane, Laurent Dercle, 27 October 2021 | Radiology: Artificial Intelligence, Vol. 3, No. 6Dual energy CT image prediction on primary tumor of lung cancer for nodal metastasis using deep learningYou-WeiWang, Chii-JenChen, Hsu-ChengHuang, Teh-ChenWang, Hsin-MingChen, Jin-YuanShih, Jin-ShingChen, Yu-SenHuang, Yeun-ChungChang, Ruey-FengChang2021 | Computerized Medical Imaging and Graphics, Vol. 91Axillary Nodal Evaluation in Breast Cancer: State of the ArtJung Min Chang, Jessica W. T. Leung, Linda Moy, Su Min Ha, Woo Kyung Moon, 21 April 2020 | Radiology, Vol. 295, No. 3Accompanying This ArticleLymph Node Metastasis Prediction from Primary Breast Cancer US Images Using Deep LearningNov 19 2019RadiologyRecommended Articles Lymph Node Metastasis Prediction from Primary Breast Cancer US Images Using Deep LearningRadiology2019Volume: 294Issue: 1pp. 19-28Preoperative Axillary US in Early-Stage Breast Cancer: Potential to Prevent Unnecessary Axillary Lymph Node DissectionRadiology2018Volume: 288Issue: 1pp. 55-63Twinkling: A Useful Adjunct for Identifying Biopsy Clips on US ImagesRadiology: Imaging Cancer2023Volume: 5Issue: 4Axillary Nodal Evaluation in Breast Cancer: State of the ArtRadiology2020Volume: 295Issue: 3pp. 500-515Ongoing Demand for Radiologists in Preoperative Axillary Lymph Node AssessmentRadiology2021Volume: 300Issue: 1pp. 55-56See More RSNA Education Exhibits Axillary Lymph Node Imaging In Breast CancerDigital Posters2021Update on the Management of the Axilla in Breast CancerDigital Posters2020Axillary Imaging in Breast Cancer: When, Who and How?Digital Posters2022 RSNA Case Collection Slow-growing cancerRSNA Case Collection2020Occult Breast CancerRSNA Case Collection2022Adenoid Cystic Carcinoma RSNA Case Collection2021 Vol. 294, No. 1 Metrics Altmetric Score PDF download