Development and validation of an artificial intelligence assisted prenatal ultrasonography screening system for trainees

Abstract Objective Fetal anomaly screening via ultrasonography, which involves capturing and interpreting standard views, is highly challenging for inexperienced operators. We aimed to develop and validate a prenatal‐screening artificial intelligence system (PSAIS) for real‐time evaluation of the quality of anatomical images, indicating existing and missing structures. Methods Still ultrasonographic images obtained from fetuses of 18–32 weeks of gestation between 2017 and 2018 were used to develop PSAIS based on YOLOv3 with global (anatomic site) and local (structures) feature extraction that could evaluate the image quality and indicate existing and missing structures in the fetal anatomical images. The performance of the PSAIS in recognizing 19 standard views was evaluated using retrospective real‐world fetal scan video validation datasets from four hospitals. We stratified sampled frames (standard, similar‐to‐standard, and background views at approximately 1:1:1) for experts to blindly verify the results. Results The PSAIS was trained using 134 696 images and validated using 836 videos with 12 697 images. For internal and external validations, the multiclass macro‐average areas under the receiver operating characteristic curve were 0.943 (95% confidence interval [CI], 0.815–1.000) and 0.958 (0.864–1.000); the micro‐average areas were 0.974 (0.970–0.979) and 0.973 (0.965–0.981), respectively. For similar‐to‐standard views, the PSAIS accurately labeled 90.9% (90.0%–91.4%) with key structures and indicated missing structures. Conclusions An artificial intelligence system developed to assist trainees in fetal anomaly screening demonstrated high agreement with experts in standard view identification.