Development and validation of a clinically applicable deep learning strategy (HONORS) for pulmonary nodule classification at CT: A retrospective multicentre study

Abstract

Purpose

To propose a practical strategy for the clinical application of deep learning algorithm, i.e., Hierarchical-Ordered Network-ORiented Strategy (HONORS), and a new approach to pulmonary nodule classification in various clinical scenarios, i.e., Filter-Guided Pyramid NETwork (FGP-NET).

Materials and methods

We developed and validated FGP-NET on a collection of 2106 pulmonary nodules on computed tomography images which combined screened and clinically detected nodules, and performed external test (n = 341). The area under the curves (AUCs) of FGP-NET were assessed. A comparison study with a group of 126 skilled radiologists was conducted. On top of FGP-NET, we built up our HONORS which was composed of two solutions. In the Human Free Solution, we used the high sensitivity operating point for screened nodules, but the high specificity operating point for clinically detected nodules. In the Human-Machine Coupling Solution, we used the Youden point.

Results

FGP-NET achieved AUCs of 0.969 and 0.847 for internal and external test. The AUCs of the subsets of the external test set ranged from 0.890 to 0.942. The average sensitivity and specificity of the 126 radiologists were 72.2 ± 15.1 % and 71.7 ± 15.5 %, respectively, while a higher sensitivity (93.3 %) but a relatively inferior specificity (64.0 %) were achieved by FGP-NET. HONORS-guided FGP-NET identified benign nodules with high sensitivity (sensitivity,95.5 %; specificity, 72.5 %) in the screened nodules, and identified malignant nodules with high specificity (sensitivity, 31.0 %; specificity, 97.5 %) in the clinically detected nodules. These nodules could be reliably diagnosed without any intervention from radiologists, via the Human Free Solution. The remaining ambiguous nodules were diagnosed with high performance, which however required manual confirmation by radiologists, via the Human-Machine Coupling Solution.

Conclusions

FGP-NET performed comparably to skilled radiologists in terms of diagnosing pulmonary nodules. HONORS, due to its high performance, might reliably contribute a second opinion, aiding in optimizing the clinical workflow.