Case Sampling for Evaluating Hospital Postoperative Morbidity in US Surgical Quality Improvement Programs

Importance US surgical quality improvement (QI) programs use data from a systematic sample of surgical cases, rather than universal review of all cases, to assess and compare risk-adjusted hospital postoperative complication rates. Given decreasing postoperative complication rates over time and the types of cases eligible for abstraction, it is unclear whether case sampling is robust for identifying hospitals with higher than expected complications. Objective To compare the assessment of hospital 30-day complication rates derived from sampling strategy used by some US surgical QI programs relative to universal review of all cases. Design, Setting, and Participants This US hospital-level analysis took place from January 1, 2016, through September 30, 2020. Data analysis was performed from July 1, 2022, through December 21, 2022. Quarterly, risk-adjusted, 30-day complication observed to expected (O-E) ratios were calculated for each hospital using the sample (n = 502 730) and universal review (n = 1 725 364). Outlier hospitals (ie, those with higher than expected mortality) were identified using an O-E ratio significantly greater than 1.0. Patients 18 years and older who underwent a noncardiac operation at US Department of Veterans Affairs (VA) hospitals with a record in the VA Surgical Quality Improvement Program (systematic sample) and the VA Corporate Data Warehouse surgical domain (100% of surgical cases) were included. Main Outcome Measure Thirty-day complications. Results Most patients in both the representative sample and the universal sample were men (90.2% vs 91.2%) and White (74.7% vs 74.5%). Overall, 30-day complication rates were 7.6% and 5.3% for the sample and universal review cohorts, respectively ( P < .001). Over 2145 hospital quarters of data, hospitals were identified as an outlier in 15.0% of quarters using the sample and 18.2% with universal review. Average hospital quarterly complication rates were 4.7%, 7.2%, and 7.4% for outliers identified using the sample only, universal review only, and concurrent identification in both data sources, respectively. For nonsampled cases, average hospital quarterly complication rates were 7.0% at outliers and 4.4% at nonoutliers. Among outlier hospital quarters in the sample, 54.2% were concurrently identified with universal review. For those identified with universal review, 44.6% were concurrently identified using the sample. Conclusion In this observational study, case sampling identified less than half of hospitals with excess risk-adjusted postoperative complication rates. Future work is needed to ascertain how to best use currently collected data and whether alternative data collection strategies may be needed to better inform local QI efforts.