Comparison of differences in cohort (forwards) and case control (backwards) methodological approaches for validation of the Hypotension Prediction Index

Background The Hypotension Prediction Index (the index) software is a machine learning algorithm that detects physiological changes that may lead to hypotension. The original validation used a case control (backwards) analysis that has been suggested to be biased. We therefore conducted a cohort (forwards) analysis and compared this to the original validation technique. Methods We conducted a retrospective analysis of data from previously reported studies. All data were analysed identically with 2 different methodologies and receiver operating characteristic curves (ROC) constructed. Both backwards and forwards analyses were performed to examine differences in area under the ROC for HPI and other haemodynamic variables to predict a MAP < 65mmHg for at least 1 minute 5, 10 and 15 minutes in advance. Results Two thousand and twenty-two patients were included in the analysis, yielding 4,152,124 measurements taken at 20 second intervals. The area-under-the-curve for the index predicting hypotension analysed by backward and forward methodologies respectively was 0.957 (95% CI, 0.947–0.964) vs 0.923 (95% CI, 0.912–0.933) 5 minutes in advance, 0.933 (95% CI, 0.924–0.942) vs 0.923 (95% CI, 0.911–0.933) 10 minutes in advance , and 0.929 (95% CI, 0.918–0.938) vs. 0.926 (95% CI, 0.914–0.937) 15 minutes in advance. No other variable had an area-under-the-curve > 0.7 except for MAP. Area-under-the-curve using forward analysis for MAP predicting hypotension 5, 10, and 15 minutes in advance was 0.932 (95% CI, 0.920–0.940), 0.929 (95% CI, 0.918–0.938), and 0.932 (95% CI, 0.921–0.940). The R 2 for the variation in the index due to MAP was 0.77. Conclusion Using an updated methodology, we found the utility of the HPI index to predict future hypotensive events is high, with an area under the receiver-operating-characteristics curve similar to that of the original validation method.