Effects of double data extraction on errors in evidence synthesis: a crossover, multicenter, investigator-blinded, randomized controlled trial

Abstract Objectives The objective was to investigate the role of double extraction in reducing data errors in evidence synthesis for pharmaceutical and non-pharmaceutical interventions. Design Crossover randomized controlled trial (RCT). Setting University and hospital with teaching programs in evidence-based medicine. Participants One hundred eligible participants were randomly assigned in a 1:1 ratio to perform data extraction tasks for either 10 RCTs of pharmaceutical interventions or 10 RCTs of non-pharmaceutical interventions, followed by a crossover pattern and a further cross-checking process (double extraction). Only data on binary adverse outcomes were extracted. Intervention Double data extraction versus single extraction. Primary and secondary outcome measures The primary outcome was the error rate before and after the cross-checking process. The secondary outcome was the absolute difference in error rates. Error rates were assessed at both the study level and the cell level. Results Error rates in the pharmaceutical and non-pharmaceutical groups were 64.65% and 59.90%, respectively, with an absolute difference of 4.75% and an odds ratio (OR) of 1.29 [95% confidence interval (CI): 1.06–1.57, P = .01] when measured at the study level. After cross-checking, error rates decreased to 44.88% and 39.54%, with the difference between the two groups remaining at 5.34%, and an OR of 1.27 (95%CI: 1.1–1.46; P < .01). Similar differences were observed when measured at the cell level. Conclusion Although double extraction reduced data errors, the error rate remained high after the process. Evidence synthesis research may consider triple data extraction to further minimize potential data errors. Trial registration number Chinese Clinical Trial Registry Center (Identifier: ChiCTR2200062206).