Measures of discrimination, reclassification, and calibration for risk prediction models: an exploration in their interrelationships and practical utility and improvement in their estimation

作者
Danielle M. Enserro
出处
期刊:Boston University - OpenBU 被引量:1
摘要

Public health practice and quality of medical care rely heavily on the accuracy, precision, and robustness of risk prediction models. Health care providers use risk prediction models to assess a patient’s risk of developing an event during a specified time frame given the patient’s specific characteristics, and subsequently recommend a course of treatment or preventative action. \n\tIn public health research, risk prediction models are often constructed with common statistical modeling techniques, such as logistic regression for binary outcomes or Cox proportional hazard regression for time-to-event outcomes, and the performance of the model is assessed through internal or external validation, or some combination. Model validation requires statistical and clinical significance and satisfactory baseline or improvement in model calibration and discrimination: calibration quantifies how close predictions are to observed outcomes while discrimination quantifies the model’s ability to distinguish correctly between events and nonevents. Measures for evaluating these qualities include (but are not limited to) Brier score, calibration-in-the-large, proportion of variation (R2), sensitivity and specificity, area under the receiver operating characteristic curve (AUC), discrimination slope, net reclassification index (NRI), integrated discrimination improvement (IDI), and decision theory analytic measures such as net benefit and relative utility. Among these measures exist several interrelationships under certain assumptions, and their estimation and interpretation is an active area of research. \n\tThe first two parts of this thesis focus on studying the empirical distributions and improving confidence interval (CI) estimation of ∆AUC, NRI, and IDI for both binary event data and time-to-event data. Through data simulation and the comparison of several CI types derived with bootstrapping techniques, we make recommendations for proper estimation in future work and apply our recommendations to real-life Framingham Heart Study data. \n\tThe third part of this thesis summarizes the many interrelationships and possible redundancies among the measures listed, extends theoretical formulas assuming normal variables for ∆AUC, NRI, and IDI from nested models to non-nested models and to Brier score, and explores the impact of varying discrimination and calibration assumptions on Yates’ and Sanders’ decomposed versions of Brier score through simulation. \n\tLastly, overall conclusions and future directions are presented at the end.

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
学霸业的应助被9979采纳,获得10
刚刚
潦草发布了新的文献求助10
刚刚
dd完成签到,获得积分20
刚刚
上官若男的应助被兔子采纳,获得10
1秒前
米歇尔发布了新的文献求助10
1秒前
搜集达人的应助被是小鱼采纳,获得10
2秒前
SciGPT的应助被taoshiren采纳,获得10
2秒前
科研通AI2S的应助被18781913856采纳,获得10
2秒前
充电宝的应助被田园采纳,获得10
5秒前
娇娇尚发布了新的文献求助10
5秒前
完美世界的应助被花花采纳,获得10
6秒前
7秒前
7秒前
SciGPT的应助被kalani采纳,获得10
7秒前
潦草完成签到,获得积分20
7秒前
8秒前
明亮白山完成签到 ,获得积分10
8秒前
祁安完成签到,获得积分10
9秒前
tian完成签到 ,获得积分10
10秒前
10秒前
小甲鱼发布了新的文献求助10
10秒前
11秒前
又听风雨完成签到,获得积分10
11秒前
11秒前
不弃发布了新的文献求助10
14秒前
napnap发布了新的文献求助10
14秒前
小学生库里完成签到,获得积分10
14秒前
王小美完成签到,获得积分10
14秒前
14秒前
taoshiren发布了新的文献求助10
15秒前
妙妙发布了新的文献求助10
15秒前
小鹅完成签到,获得积分10
15秒前
Lucas的应助被机灵花生采纳,获得10
15秒前
xiaoyu发布了新的文献求助10
16秒前
16秒前
丘比特的应助被汌舟采纳,获得10
17秒前
彭于晏的应助被哇哇哇采纳,获得10
17秒前
ding完成签到,获得积分10
17秒前
dd0926完成签到 ,获得积分10
17秒前
张倩发布了新的文献求助10
18秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
Sex, Aging, and Death in a Medieval Medical Compendium: Trinity College Cambridge MS R.14.52, Its Texts, Language, and Scribe 500
高分子基功能界面水蒸发材料与应用研究 500
外场增强的水汽发电研究 500
基于结构调控的功能化水诱导发电材料与器件研究 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8189382
求助须知:如何正确求助?哪些是违规求助? 9576691
关于积分的说明 21849410
捐赠科研通 7787318
什么是DOI,文献DOI怎么找? 3435538
关于科研通互助平台的介绍 2559748
邀请新用户注册赠送积分活动 2482509