概化理论
外部有效性
交叉验证
计算机科学
预测建模
统计
数据挖掘
计量经济学
数学
人工智能
机器学习
作者
Toshihiko Takada,Steven W J Nijman,Spiros Denaxas,Kym I E Snell,Alicia Uijl,Tri‐Long Nguyen,Folkert Wouter Asselbergs,Thomas P. A. Debray
标识
DOI:10.1016/j.jclinepi.2021.03.025
摘要
OBJECTIVE: To illustrate how to evaluate the need of complex strategies for developing generalizable prediction models in large clustered datasets. STUDY DESIGN AND SETTING: We developed eight Cox regression models to estimate the risk of heart failure using a large population-level dataset. These models differed in the number of predictors, the functional form of the predictor effects (non-linear effects and interaction) and the estimation method (maximum likelihood and penalization). Internal-external cross-validation was used to evaluate the models' generalizability across the included general practices. RESULTS: Among 871,687 individuals from 225 general practices, 43,987 (5.5%) developed heart failure during a median follow-up time of 5.8 years. For discrimination, the simplest prediction model yielded a good concordance statistic, which was not much improved by adopting complex strategies. Between-practice heterogeneity in discrimination was similar in all models. For calibration, the simplest model performed satisfactorily. Although accounting for non-linear effects and interaction slightly improved the calibration slope, it also led to more heterogeneity in the observed/expected ratio. Similar results were found in a second case study involving patients with stroke. CONCLUSION: In large clustered datasets, prediction model studies may adopt internal-external cross-validation to evaluate the generalizability of competing models, and to identify promising modelling strategies.
科研通智能强力驱动
Strongly Powered by AbleSci AI