External Validation of an Artificial Intelligence Triaging System for Chest X-Rays: A Retrospective Independent Clinical Study

Background: Chest radiography (CXR) is the most frequently performed radiological exam worldwide, yet reporting backlogs due to radiologist shortages remain critical in emergency care. Artificial intelligence triage systems may alleviate this challenge by differentiating normal from abnormal studies and prioritizing urgent cases. This study aimed to externally validate TRIA, a commercial AI-powered CXR triage algorithm (NeuralMed, São Paulo, Brazil). Methods: TRIA uses a two-stage deep learning approach: an image segmentation module to isolate the thoracic region, followed by a classification model trained to recognize common cardiopulmonary pathologies. The system was trained on 275,399 CXRs from multiple public and private datasets. External validation was performed retrospectively on 1,045 CXRs (568 normal, 477 abnormal) from a teaching university hospital. Ground truth was derived from radiologist reports using a large language model–assisted extraction pipeline; a subset of 300 reports was independently reviewed by a radiologist (accuracy 0.98; 95% confidence intervals (CI) 0.978–0.988). Four ensemble decision strategies for abnormality detection were compared. Performance metrics included sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve (AUROC) with 95% CI). Results: The general abnormality classifier achieved strong performance (AUROC 0.911). Individual pathology models for cardiomegaly, pneumothorax, and effusion showed excellent results (AUROC 0.968, 0.955, and 0.935, respectively). The weighted ensemble demonstrated the best balance, with accuracy 0.854 (95% CI 0.831–0.874), sensitivity 0.845 (0.810–0.875), specificity 0.861 (0.830–0.887), and AUROC 0.927 (0.911–0.940). Sensitivity-prioritized methods (>0.92) produced lower specificity (<0.69). False negatives were mainly subtle or equivocal cases, although many were still flagged abnormal by the general classifier. Conclusions: TRIA achieved robust and balanced accuracy in distinguishing normal from abnormal CXRs. Integration into clinical workflows could reduce reporting delays, prioritize urgent cases, and improve patient safety. These findings support its clinical utility and warrant prospective multicenter validation.