Early clinical evaluation of a machine-learning system for risk prediction of trauma-induced coagulopathy in the prehospital setting

BACKGROUND: Early intervention in patients with major traumatic injuries is critical. Decision support can improve clinicians' ability to identify high-risk patients. The aim of this study was to compare the performance of a machine-learning (ML) decision support system to that of expert clinicians and to assess the ML system's impact on augmenting human risk prediction after injury in the prehospital phase of care. METHODS: This early clinical evaluation study compared a ML risk prediction system to expert clinicians in assessing a patient's risk of trauma-induced coagulopathy (TIC). The study was conducted between 1 January 2019 and 31 June 2019 at two air ambulance sites in the south of England. The ML system used a Bayesian Network algorithm to predict TIC. Comparisons in predictive performance were made first between expert clinicians and the ML system and second, between expert clinicians and expert clinicians exposed to the ML system's outputs. RESULTS: Overall, 51 expert clinicians were enrolled in the study and 184 patient assessments from 135 patients were analysed. The median age of included patients was 31 years old (IQR 23, 47), 75% were male and median Injury Severity Score 17 (IQR 9, 34). 62 patients (46%) received blood within 4 hours of injury and 26 patients (19%) developed TIC. The ML system did not outperform expert clinicians in discriminating between patients with and without TIC (area under the curve (AUC) ML: 0.87 (95% CI 0.79, 0.95) vs AUC clinician: 0.83 (95% CI 0.74, 0.92), p=0.330)). Calibration and overall accuracy of the ML system were superior. Expert clinicians' risk prediction, when augmented by the ML system, showed potential for improvement compared with unassisted human performance. CONCLUSIONS: Early after injury, an ML system performs well compared with expert prehospital clinicians in the prediction of TIC and blood transfusion. The study suggests that ML systems may augment clinical risk prediction in trauma.