Explaining alerts from a pediatric risk prediction model using clinical text

Abstract Objective Risk prediction models are used in hospitals to identify pediatric patients at risk of clinical deterioration, enabling timely interventions and rescue. The objective of this study was to develop a new explainer algorithm that uses a patient’s clinical notes to generate text-based explanations for risk prediction alerts. Materials and Methods We conducted a retrospective study of 39 406 patient admissions to the American Family Children’s Hospital at the University of Wisconsin-Madison (2009-2020). The pediatric Calculated Assessment of Risk and Triage (pCART) validated risk prediction model was used to identify children at risk for deterioration. A transformer model was trained to use clinical notes from the 12-hour period preceding each pCART score to predict whether a patient was flagged as at risk. Then, label-aware attention highlighted text phrases most important to an at-risk alert. The study cohort was randomly split into derivation (60%) and validation (20%) data, and a separate test (20%) was used to evaluate the explainer’s performance. Results Our pCART Explainer algorithm performed well in discriminating at-risk pCART alert vs no alert (c-statistic 0.805). Sample explanations from pCART Explainer revealed clinically important phrases such as “rapid breathing,” “fall risk,” “distension,” and “grunting,” thereby demonstrating excellent face validity. Discussion The pCART Explainer could quickly orient clinicians to the patient’s condition by drawing attention to key phrases in notes, potentially enhancing situational awareness and guiding decision-making. Conclusion We developed pCART Explainer, a novel algorithm that highlights text within clinical notes to provide medically relevant context about deterioration alerts, thereby improving the explainability of the pCART model.