Validity of cluster headache diagnoses in an electronic health record data repository

BACKGROUND: Cluster headache (CH) is uncommon, so electronic searches of large medical record databases provide an important opportunity to identify a sufficient number of patients for research, patient registries, or quality improvement work. However, the accuracy of CH diagnoses recorded in electronic health record (EHR) databases is unknown. The Research Patient Data Registry (RPDR) warehouses information from EHR of two academic medical centers in Boston. We assessed the validity of CH diagnoses in the RPDR as well as the sensitivity of International Classification of Headache Disorders (ICHD) diagnostic criteria in relation to the gold standard of expert clinician diagnosis of CH. METHODS: In 2013, we queried RPDR to identify all patients diagnosed with CH diagnosed by headache specialists from 2002 to 2012. We sought to determine the positive predictive value (PPV) of an EHR diagnosis of CH relative to a headache expert's clinical impression as the gold standard. We also calculated the sensitivity of ICHD-2 CH criteria compared with the gold standard of expert-diagnosed CH cases. RESULTS: RPDR queries identified 170 patients with a diagnosis of CH. Two researchers carried out a detailed chart review searching for information to support or refute this diagnosis. In 58 cases, the diagnosis of CH was determined to be inaccurate due to data entry errors. The PPV of an RPDR diagnosis of CH was 63% (95% CI 54-71%) and the sensitivity of ICHD-2 criteria compared with the gold standard of expert diagnosis was 77% (95% CI 69-82%). CONCLUSIONS: The PPV of EHR diagnoses of CH is modest. Data entry errors are common, and only about three-fourths of CH cases diagnosed by expert clinicians meet ICHD criteria for CH. Thus, EHR searches for patients with CH frequently will result in false positives. This has implications for activities such as research, quality improvement efforts, or disease registries that rely on EHR searches to identify patients. Further work is needed to develop high quality phenotypic algorithms so that the research potential of EHRs can be realized.