Association of Statistical Methodology and Design in Preclinical Animal Studies With Successful Translation Into Clinical Phase 2 Trials

BACKGROUND AND OBJECTIVES: Preclinical-to-clinical translational failure is common in neurologic research. Reasons for translational failure are complex and multifaceted, but it is increasingly recognized that translational decision making may be influenced by preclinical work that is overly optimistic, due to a lack of scientific rigor. Previous research has demonstrated that statistical errors are common in preclinical research, yet results of preclinical null hypothesis significance testing contribute to the body of evidence used to evaluate whether a new intervention is a candidate for translation. However, no empirical investigations exist yet that compare positive and negative clinical trials on the statistical soundness of contributing studies. The aim of this study was to explore, for 3 neurologic indications (multiple sclerosis, Parkinson disease, epilepsy), whether negative clinical trials show a higher prevalence of statistical misapplication in preceding animal experiments, compared with positive human trials. METHODS: A modified meta-research, case-control design with multistep systematic search was used. Phase 2 clinical trials (completed January 1, 2010-October 31, 2020) were identified on ClinicalTrials.gov. Best practice methods were used to systematically search MEDLINE and Embase for animal experiments preceding the start of each human trial, for each intervention and disease. Statistical reporting and decision-making data were gathered from animal articles by collectors blinded to human trial outcome. Rates of statistical mistakes were compared between animal articles preceding positive vs negative human trials using weighted percentages and CIs. RESULTS: The final sample included 24 trials (8 negative) and 70 associated rodent studies. Animal studies preceding negative human trials had higher rates of misapplication of solely cross-sectional statistical tests to longitudinal data (93% [95% CI 83-100] vs 66% [95% CI 47-82]) and of the use of plots that concealed continuous data distributions (98% [95% CI 95-100] vs 71% [95% CI 51-91]), indicating possible mishandling of continuous data. DISCUSSION: Statistical practice in animal studies was generally poor or not well reported, with rates of some statistical mistakes occurring more frequently before negative trials than positive trials. This was an exploratory study with a highly selected sample, yet it suggests that translational decision making should include evaluation of the quality of preclinical statistical practice.