Exploring the Relationships Between USMLE Performance and Disciplinary Action in Practice: A Validity Study of Score Inferences From a Licensure Examination

Purpose Physicians must pass the United States Medical Licensing Examination (USMLE) to obtain an unrestricted license to practice allopathic medicine in the United States. Little is known, however, about how well USMLE performance relates to physician behavior in practice, particularly conduct inconsistent with safe, effective patient care. The authors examined the extent to which USMLE scores relate to the odds of receiving a disciplinary action from a U.S. state medical board. Method Controlling for multiple factors, the authors used non-nested multilevel logistic regression analyses to estimate the relationships between scores and receiving an action. The sample included 164,725 physicians who graduated from U.S. MD-granting medical schools between 1994 and 2006. Results Physicians had a mean Step 1 score of 214 (standard deviation [SD] = 21) and a mean Step 2 Clinical Knowledge (CK) score of 213 (SD = 23). Of the physicians, 2,205 (1.3%) received at least one action. Physicians with higher Step 2 CK scores had lower odds of receiving an action. A 1-SD increase in Step 2 CK scores corresponded to a decrease in the chance of disciplinary action by roughly 25% (odds ratio = 0.75; 95% CI = 0.70–0.80). After accounting for Step 2 CK scores, Step 1 scores were unrelated to the odds of receiving an action. Conclusions USMLE Step 2 CK scores provide useful information about the odds a physician will receive an official sanction for problematic practice behavior. These results provide validity evidence supporting current interpretation and use of Step 2 CK scores.