Real-time artificial intelligence for detection of Fetal Intracranial malformations in Ultrasonic images: A multicenter retrospective diagnostic study

Objective: To develop an artificial intelligence (AI) model to detect congenital central nervous system (CNS) malformations in fetal cerebral-cranial ultrasound images, and to assess the efficacy of this algorithm in improving clinical doctors’ diagnostic performance. Design: Retrospective, multicenter, diagnostic study Setting: Three Chinese hospitals Population: a cohort of 2397 fetuses with CNS malformations and 11316 normal fetuses. Methods: AI model was developed by training on 37450 images from 15264 fetuses and testing on 812 images from 449 fetuses. Three groups of doctors (trainee, competent, expert) were equipped with the AI system to test its enhancement of diagnosis performance. Main outcome measures: Diagnostic performance of AI model and that of doctors. Comparison of performance between AI model and doctors, and doctors with and without AI assistance. Results: The performance of AI model was comparable to that of expert in identifying 12 types of CNS malformations in terms of accuracy 79.8% (95% CI 77.0-82.6% ) versus 78.9% (95% CI 75.2-85.2% ), sensitivity 78.4% (75.3-81.3%) versus 77.5% (73.7-81.4%) , specificity of 94.4% (86.2-98.4%) versus 93.0% (84.1-100.0%), and AUC 0.864 (0.833-0.895) versus 0.853 (0.800-0.905). This AI model improved doctors’ diagnostic performances, the trainee group received maximum improvement, whose diagnostic performance advanced to the level of expert group in terms of accuracy (80.2%, 95% CI 75.0-85.3% ) and AUC (0.872, 95% CI 0.861-0.882 ). Conclusions: Our AI system achieved a high diagnostic performance comparable with that of experienced doctors and can support unexperienced doctors by improving their diagnostic accuracy to an expert-level.