An artificial intelligence ultrasound system’s ability to distinguish benign from malignant follicular-patterned lesions

Objectives To evaluate the application value of a generally trained artificial intelligence (AI) automatic diagnosis system in the malignancy diagnosis of follicular-patterned thyroid lesions (FPTL), including follicular thyroid carcinoma (FTC), adenomatoid hyperplasia nodule (AHN) and follicular thyroid adenoma (FTA) and compare the diagnostic performance with radiologists of different experience levels. Methods We retrospectively reviewed 607 patients with 699 thyroid nodules that included 168 malignant nodules by using postoperative pathology as the gold standard, and compared the diagnostic performances of three radiologists (one junior, two senior) and that of AI automatic diagnosis system in malignancy diagnosis of FPTL in terms of sensitivity, specificity and accuracy, respectively. Pairwise t-test was used to evaluate the statistically significant difference. Results The accuracy of the AI system in malignancy diagnosis was 0.71, which was higher than the best radiologist in this study by a margin of 0.09 with a p-value of 2.08×10 -5 . Two radiologists had higher sensitivity (0.84 and 0.78) than that of the AI system (0.69) at the cost of having much lower specificity (0.35, 0.57 versus 0.71). One senior radiologist showed balanced sensitivity and specificity (0.62 and 0.54) but both were lower than that of the AI system. Conclusions The generally trained AI automatic diagnosis system can potentially assist radiologists for distinguishing FTC from other FPTL cases that share poorly distinguishable ultrasonographical features.