摘要
Introduction: Indications for blood transfusion in gastrointestinal hemorrhage are frequently under scrutiny, but guidelines and trials tend to focus on solitary data points, like hemoglobin measurements. Additionally, few studies have explored balanced resuscitation techniques in this population. Reinforcement learning is an artificial intelligence technique that is making headlines with superhuman levels of performance in Markovian decision-making problems. This machine learning framework is well-suited to blood product resuscitation. Methods: Using the MIMIC dataset, we collected structured clinical data on 4,630 ICU admissions for both upper and lower gastrointestinal hemorrhage from 2008-2019. Observations were discretized to 1-hour time intervals, and K-mean clustering was performed for patient state representation. A Gastrointestinal Bleeding Reinforcement Learning agent, GaBRieL, was trained to recommend blood product administration using offline Q-learning and evaluated on a 20% held-out validation set. At a given timestep, blood product administration was treated as a binary decision for each of packed red blood cells (PRBC), fresh frozen plasma (FFP), and platelets (Plts). Results: When compared with clinicians acting on unseen validation data, GaBRieL recommended transfusion of PRBC at higher hemoglobin levels [8.91 vs 8.06, p<.0001], FFP at lower INR levels [1.61 vs 2.23, p<.0001], and Plts at higher platelet levels [130 vs 66.5, p<.0001]. GaBRieL also initiated first blood product transfusion an average of eight hours earlier than did clinicians [p<.0001]. Higher in-hospital mortality was associated with failure to transfuse recommended PRBC [p<.0001] and FFP [p=.03]. Overall, in-hospital mortality was lower when clinician action matched GaBRieL’s recommendation [p<.0001]. Conclusions: Our reinforcement learning agent, GaBRieL, learned an optimal transfusion strategy in GI hemorrhage that differed from clinician action in important ways. GaBRieL recommended less restrictive transfusion of PRBC, FFP, and Plts, with a strong suggestion of lower associated mortality. These findings are both hypothesis-generating for studies investigating balanced transfusion strategies in non-trauma patients and suggest that reinforcement learning may play a role in guiding resuscitation at the bedside.