Statistical peer review of surgical articles from the joint viewpoint of a medical biostatistician and a surgeon scientist

Most medical studies, whether basic science experiments, randomized controlled trials, or observational investigations, entail numbers subject to statistical analysis. Yet statistical reviews remain relatively rare among biomedical journals. 1 Hardwicke T.E. Goodman S.N. How often do leading biomedical journals use statistical experts to evaluate statistical methods? The results of a survey. PLoS One. 2020; 15 (e0239598–e0239598) Google Scholar We present a guide for non-statisticians to perform statistical reviews of surgical articles (Table I). This guide is based on existing resources 2 Nature PortfolioStatistical guidelines. Nature Submission Guidelines. https://www.nature.com/srep/author-instructions/submission-guidelines#statistical-guidelinesDate accessed: September 3, 2021 Google Scholar , 3 Mansournia M.A. Collins G.S. Nielsen R.O. et al. A Checklist for statistical Assessment of Medical Papers (the CHAMP statement): explanation and elaboration. Br J Sports Med. 2021; 55: 1009-1017 Google Scholar , 4 UK Equator CentreThe EQUATOR Network: Enhancing the QUAlity and Transparency Of health Research. https://www.equator-network.org/Date accessed: September 3, 2021 Google Scholar and the joint experience of a medical biostatistician and a surgeon scientist who have peer-reviewed thousands of surgical publications. In the following paragraphs, we emphasize common statistical issue that require attention. Table IChecklist for statistical assessment of surgical papers Important statistical issues Request review by statistician Reporting standard Reporting standard described, followed, and checklist provided Study design Appropriate study design used to achieve the objective(s)/hypothesis(es) •If hypothesis is non-inferiority/equivalence, the hypothesis defines well-based non-inferiority margin, and the study design and power calculations are appropriate for this design •Adequate comparator/control group •Factorial or other complex designs •Randomized controlled trial •Non-inferiority or Equivalence study Sample Source(s) of subjects/data appropriate for study purpose, contains all elements required to respond the research question, and is/are appropriately described Sampling technique appropriately described Power/sample size appropriately described •Non-inferiority or Equivalence study •Superiority study missing >1 of the 4 sample size/power elements (confidence, power, minimum difference to be detected, comparator values) •Post-hoc sample size using the difference observed in the current study Actual study sample •In/exclusion criteria clearly defined •CONSORT-like diagram included •Data exclusions are stated/explained and impact on results are explored •N reported at the start of the study, for each data set and for each analysis •Discrepancies in value of N between analyses clearly explained/ justified •Missing data are explained, and impact on findings minimized/explained •If longitudinal study: follow-up/ response rate •If survey: response rate •Discrepancy in N between analyses •Missing data >15% or clearly not at random (eg, higher missing rate in patients with outcome) •Longitudinal studies Statistical analysis Appropriate measure(s) of central tendency (eg, mean or median)/dispersion (eg, standard deviation or range) Unit of analysis given for all comparisons •Multiple data by subject (limbs, eyes, wounds, infections, readmission, etc.) Alpha level given for all statistical tests Tests clearly identified as one or two-tailed •One-tailed tests Adequate univariate/bivariate statistical analyses used/described •If paired data, appropriate paired tests used •Assumptions of tests applied met (particular attention paid to skewed data or small sample sizes) •In prognostic or diagnostic studies, predictive value of a positive and negative were presented in addition to sensitivity and specificity Adequate multivariate statistical analyses described •If a multicenter study: clustering effects by facility/region/other accounted for using appropriate analytic approaches •If matching (propensity score or traditional matching) was used, center was included in the matching, and balance diagnostics were described •Appropriate measures of model performance were presented •Signs of poor model fit or model overfit were addressed •If the outcome is subject survivor-bias, appropriate analytic approaches were used •Multicenter studies •Matching (propensity score or other matching methods) •Lack of model performance measures in multivariate analysis •Signs of poor model fit (eg, large confidence intervals, effect with unplausible direction) •Suspicion of overfitting (>1 variable per 10 subjects with the outcome) •Outcome subject to survivor-bias Appropriate measure(s) of effect (eg, relative risks for outcomes with frequency >20%) Confounding and bias explored and minimized •All confounders/covariates presented in full in multivariate models Adjustments made for multiple testing explained Actual P values are given for primary analyses and confidence intervals (or credible intervals in case of Bayesian analysis) given for the main results Unusual/complex statistical methods clearly explained Conclusion Conclusion drawn from the statistical analysis is justified CONSORT, Consolidated Standards of Reporting Trials. Open table in a new tab CONSORT, Consolidated Standards of Reporting Trials.