Statistics in biology: a survey of the three major multidisciplinary journals

Abstract This article presents the results of a survey of the three major multidisciplinary journals (Nature, Proceedings of the National Academy of Science, Science). Fifty articles involving experiments necessitating the agreement of an ethics committee were searched in each of the three journals to June 30th 2023. Because PNAs announced that a Statistical Review Committee was created and working since September 2023, a further set of 50 articles was retrieved and analyzed from 2024 January 1st. The following items were checked in articles: an explicit statement of approval from an animal ethics committee, the calculation of the appropriate samples size needed and how randomization and blinding were performed, the minimum sample size reported, the presence of repeats, and the methods used to limit type I and type II errors. No clear experimental design was fully reported in any article. The major problems were 1) extremely small sample sizes (<4/group) in nearly half of the articles, 2) confusion between biological repeats and technical replications, and 3) lack of correction for multiple comparisons. These errors led to major inflation of type I and type II errors. In conclusion, only 10 percent of the articles analyzed presented correct statistical methodology.