Internal and external validation of machine learning–assisted prediction models for mechanical ventilation–associated severe acute kidney injury

Abstract

Background

Currently, very few preventive or therapeutic strategies are used for mechanical ventilation (MV)-associated severe acute kidney injury (AKI).

Objectives

We developed clinical prediction models to detect the onset of severe AKI in the first week of intensive care unit (ICU) stay during the initiation of MV.

Methods

A large ICU database Medical Information Mart for Intensive Care IV (MIMIC-IV) was analysed retrospectively. Data were collected from the clinical information recorded at the time of ICU admission and during the initial 12 h of MV. Using univariate and multivariate analyses, the predictors were selected successively. For model development, two machine learning algorithms were compared. The primary goal was to predict the development of AKI stage 2 or 3 (AKI-23) and AKI stage 3 (AKI-3) in the first week of patients' ICU stay after initial 12 h of MV. The developed models were externally validated using another multicentre ICU database (eICU Collaborative Research Database, eICU) and evaluated in various patient subpopulations.

Results

Models were developed using data from the development cohort (MIMIC-IV: 2008–2016; n = 3986); the random forest algorithm outperformed the logistic regression algorithm. In the internal (MIMIC-IV: 2017–2019; n = 1210) and external (eICU; n = 1494) validation cohorts, the incidences of AKI-23 were 154 (12.7%) and 119 (8.0%), respectively, with areas under the receiver operator characteristic curve of 0.78 (95% confidence interval [CI]: 0.74–0.82) and 0.80 (95% CI: 0.76–0.84); the incidences of AKI-3 were 81 (6.7%) and 67 (4.5%), with areas under the receiver operator characteristic curve of 0.81 (95% CI: 0.76–0.87) and 0.80 (95% CI: 0.73–0.86), respectively.

Conclusions

Models driven by machine learning and based on routine clinical data may facilitate the early prediction of MV-associated severe AKI. The validated models can be found at: https://apoet.shinyapps.io/mv_aki_2021_v2/.