Inclusion of Retracted Studies in Systematic Reviews and Meta-Analyses of Interventions

Importance Retractions are rising in the scientific literature, increasing the risk of reusing unreliable results. Objectives To identify reports of systematic reviews that included retracted studies in their meta-analyses, and to assess the impact of these retracted studies on the results. Design, Setting, and Participants In this systematic review and meta-analysis the Feet of Clay Detector tool was searched to identify all systematic reviews that reported at least 1 meta-analysis including at least 1 retracted study and were published in the 25 highest impact factor journals in medicine, general and internal, from January 2013 to April 2024. All effect estimates where the retracted study contributed to the analysis were identified. For each meta-analysis, the summary effect, including all studies and excluding retracted studies was calculated. The search was conducted on April 8, 2024. Results Overall, 61 systematic reviews were identified that included retracted studies in their meta-analyses. Of these, 11 (18%) have been republished, retracted, or withdrawn. Data were extracted from 50 systematic reviews that included a total of 62 retracted studies. Retraction occurred after the publication in 37 systematic reviews (74%). Overall, 173 meta-analyses including the retracted study were identified; 70 of them (40%) were primary outcomes. One-hundred sixty-six meta-analyses were recalculated. Overall, 160 (96%) of the 166 recalculated effect estimates were within the CIs of the original effect. After exclusion of the retracted study, the statistical significance of the results changed in 18 meta-analyses (11%). The rate of evolution between effect estimates with and without retracted studies for 163 meta-analyses overall were calculated, including 64 addressing the primary outcomes of the systematic reviews. For primary outcomes (n = 64), the recalculated effect estimates changed by at least 10% in 27 meta-analyses (42%), 30% in 16 (25%), and 50% in 12 (19%). Overall (n = 163), effect estimates changed by at least 10% in 57 meta-analyses (35%), 30% in 31 (19%), and 50% in 23 (14%). Conclusions and Relevance This study found that retracted studies have been included in systematic reviews and meta-analyses, with retractions occurring mainly after the publication of the systematic review. The inclusion of retracted studies can impact the results and interpretation of reviews. Quality control measures should be implemented to prevent the dissemination of unreliable data in scientific literature.