“Dynamic Ensemble, Then Knowledge Distillation”: A SHAP-Driven Two-Stage Framework for Sepsis Mortality Prediction

Dynamic Ensemble Learning (DEL)-based mortality prediction models offer promising utility for monitoring sepsis progression. Existing DEL methods rely on unsupervised techniques, such as bootstrap sampling or feature partitioning, to generate training data subsets, failing to capture the inherent heterogeneity in the relationships between sepsis phenotypes and associated mortality risks. Moreover, combining heterogeneous base models leads to a loss of model explainability. To this end, we propose a novel SHapley Additive exPlanations (SHAP)-driven Dynamic Ensemble, then Knowledge Distillation (DEKD) two-stage framework for sepsis mortality prediction. DEKD first clusters patients based on the SHAP values, which reflect the mapping relationship between their physiological features and mortality risk, and then builds base models on these patient clusters. A weighted distance-based ensemble strategy is further adopted to adaptively aggregate predictions from the base models. In the second stage, we leverage knowledge distillation to extract the patterns from the ensemble of base models into a student model, thereby generating overall explanation and improving predictive performance. Experiments conducted on the MIMIC-III dataset demonstrate the effectiveness of the proposed two-stage strategy. DEKD achieves AUROC of 0.955 (48-hour mortality prediction) and 0.924 (in-hospital mortality prediction), exhibiting significant improvement in diversity measure compared to benchmark methods.