计算机科学
机器学习
公制(单位)
排名(信息检索)
人工智能
黑匣子
集合(抽象数据类型)
光学(聚焦)
分割
简单(哲学)
数据挖掘
控制(管理)
程序设计语言
哲学
光学
物理
运营管理
经济
认识论
作者
Stephen Bates,Anastasios N. Angelopoulos,Lihua Lei,Jitendra Malik,Michael I. Jordan
出处
期刊:Journal of the ACM
[Association for Computing Machinery]
日期:2021-09-30
卷期号:68 (6): 1-34
被引量:534
摘要
While improving prediction accuracy has been the focus of machine learning in recent years, this alone does not suffice for reliable decision-making. Deploying learning systems in consequential settings also requires calibrating and communicating the uncertainty of predictions. To convey instance-wise uncertainty for prediction tasks, we show how to generate set-valued predictions from a black-box predictor that controls the expected loss on future test points at a user-specified level. Our approach provides explicit finite-sample guarantees for any dataset by using a holdout set to calibrate the size of the prediction sets. This framework enables simple, distribution-free, rigorous error control for many tasks, and we demonstrate it in five large-scale machine learning problems: (1) classification problems where some mistakes are more costly than others; (2) multi-label classification, where each observation has multiple associated labels; (3) classification problems where the labels have a hierarchical structure; (4) image segmentation, where we wish to predict a set of pixels containing an object of interest; and (5) protein structure prediction. Last, we discuss extensions to uncertainty quantification for ranking, metric learning, and distributionally robust learning.
科研通智能强力驱动
Strongly Powered by AbleSci AI