计算机科学
人工智能
医学
临床实习
放射科
自动化方法
梅德林
医学物理学
医学影像学
核医学
作者
Maximilian Frederik Russe,Anna Fink,Carl P. Simon,Stephan Rau,Kai Falko Kästingschäfer,Fabian Bamberg,Alexander Rau
标识
DOI:10.1097/rli.0000000000001272
摘要
BACKGROUND: Despite the growing number of artificial intelligence (AI)-based applications used in radiology, no structured framework exists to assess their case-level reliability or to document overridden outputs in reports. PURPOSE: To develop and evaluate the Artificial Intelligence Reporting and Data System (AI-RADS), a structured framework for an objective, case-level assessment of AI output reliability, clinical utility, and recommended actions in radiology. MATERIALS AND METHODS: The AI-RADS framework was tested in a retrospective, multireader study. Here, 5 board-certified radiologists independently evaluated 350 cases processed by 7 representative AI applications for image-based and generative tasks. Each case was assigned one of 5 AI-RADS categories, applicable modifiers, and an independent correctness rating as a reference. Interreader agreement was quantified using Krippendorff's α with 95% CIs. RESULTS: Substantial interreader agreement was observed for the core AI-RADS categories in both image-based (Krippendorff's α=0.87; 95% CI: 0.83-0.91) and generative AI tasks (Krippendorff's α=0.93; 95% CI: 0.91-0.95). Reader-assigned correctness aligned well with AI-RADS categories 1 to 2, which indicate outputs suitable for integration into clinical workflows. Outputs rated as "incorrect" were predominantly assigned to categories 4 to 5, warranting override or removal from display. CONCLUSION: AI-RADS provides a structured framework for the case-level evaluation of AI output reliability, clinical utility, and consequences for report communication. This multireader study demonstrated substantial interreader agreement and applicability across various AI applications.
科研通智能强力驱动
Strongly Powered by AbleSci AI