Receiver Operating Characteristic (ROC) Curves: The Basics and Beyond

Diagnostic tests and clinical prediction rules are frequently used to help estimate the probability of a disease or outcome. How well a test or rule distinguishes between disease or no disease (discrimination) can be measured by plotting a receiver operating characteristic (ROC) curve and calculating the area under it (AUROC). In this paper, we review the features of ROC curves and interpretation of ROC curves and AUROC values. We highlight 5 underappreciated features of ROC curves: (1) the slope of the ROC curve over a test result interval is the likelihood ratio for that interval; (2) the optimal cutoff for calling a test positive depends not only on the shape of the ROC curve, but also on the pretest probability of disease and relative harms of false-positive and false-negative results; (3) the AUROC measures discrimination only, not the accuracy of the predicted probabilities; (4) the AUROC is not a good measure of discrimination if the slope of the ROC curve is not consistently decreasing; and (5) the AUROC can be increased by including a large number of people correctly identified as being at very low risk for the outcome of interest. We illustrate this last concept using 3 published studies.