A Comparative Analysis of Sepsis Identification Methods in an Electronic Database*

Objectives: To evaluate the relative validity of criteria for the identification of sepsis in an ICU database. Design: Retrospective cohort study of adult ICU admissions from 2008 to 2012. Setting: Tertiary teaching hospital in Boston, MA. Patients: Initial admission of all adult patients to noncardiac surgical ICUs. Interventions: Comparison of five different algorithms for retrospectively identifying sepsis, including the Sepsis-3 criteria. Measurements and Main Results: 11,791 of 23,620 ICU admissions (49.9%) met criteria for the study. Within this subgroup, 59.9% were suspected of infection on ICU admission, 75.2% of admissions had Sequential Organ Failure Assessment greater than or equal to 2, and 49.1% had both suspicion of infection and Sequential Organ Failure Assessment greater than or equal to 2 thereby meeting the Sepsis-3 criteria. The area under the receiver operator characteristic of Sequential Organ Failure Assessment (0.74) for hospital mortality was consistent with previous studies of the Sepsis-3 criteria. The Centers for Disease Control and Prevention, Angus, Martin, Centers for Medicare & Medicaid Services, and explicit coding methods for identifying sepsis revealed respective sepsis incidences of 31.9%, 28.6%, 14.7%, 11.0%, and 9.0%. In-hospital mortality increased with decreasing cohort size, ranging from 30.1% (explicit codes) to 14.5% (Sepsis-3 criteria). Agreement among the criteria was acceptable (Cronbach’s alpha, 0.40–0.62). Conclusions: The new organ dysfunction-based Sepsis-3 criteria have been proposed as a clinical method for identifying sepsis. These criteria identified a larger, less severely ill cohort than that identified by previously used administrative definitions. The Sepsis-3 criteria have several advantages over prior methods, including less susceptibility to coding practices changes, provision of temporal context, and possession of high construct validity. However, the Sepsis-3 criteria also present new challenges, especially when calculated retrospectively. Future studies on sepsis should recognize the differences in outcome incidence among identification methods and contextualize their findings according to the different cohorts identified.