Trial characteristics and treatment effect estimates in randomized controlled trials of Chinese herbal medicine: A meta-epidemiological study

Previously published meta-epidemiological studies focused on Western medicine have identified some trial characteristics that impact the treatment effect of randomized controlled trials (RCTs). Nevertheless, it remains unclear if similar associations exist in RCTs on Chinese herbal medicine (CHM). Further, Chinese medicine-related characteristics have not been explored yet. To investigate trial characteristics related to treatment effect estimates on CHM RCTs. This meta-epidemiological study searched 5 databases for systematic reviews on CHM treatment published between January 2011 and July 2021. An eligible systematic review should only include RCTs of CHM and conduct at least one meta-analysis. Two reviewers independently conducted data extraction on general characteristics of systematic reviews, meta-analyses and included RCTs. They also assessed the risk of bias of RCTs using the Cochrane risk of bias tool. A two-step method was used for data analyses. The ratio of odds ratios (ROR) and difference in standardized mean differences (dSMD) with 95% confidence interval (CI) were applied to present the difference in effect estimates for binary and continuous outcomes, respectively. Ninety-one systematic reviews, comprising 1338 RCTs were identified. For binary outcomes, RCTs incorporated with syndrome differentiation (ROR: 1.23; 95% CI: [1.07, 1.39]), adopting Chinese medicine formula (ROR: 1.19; 95% CI: [1.03, 1.34]), with low risk of bias on incomplete outcome data (ROR: 1.29; 95% CI: [1.06, 1.52]) and selective outcome reporting (ROR: 1.12; 95% CI: [1.01, 1.24]), as well as a trial size ≥ 100 (ROR: 1.23; 95% CI: [1.04, 1.42]) preferred to show larger effect estimates. As for continuous outcomes, RCTs with Chinese medicine diagnostic criteria (dSMD: 0.23; 95% CI: [0.06, 0.41]), judged as high/unclear risk of bias on allocation concealment (dSMD: -0.70; 95% CI: [-0.99, -0.42]), with low risk of bias on incomplete outcome data (dSMD: 0.30; 95% CI: [0.18, 0.43]), conducted at a single center (dSMD: -0.33; 95% CI: [-0.61, -0.05]), not using intention-to-treat analysis (dSMD: -0.75; 95% CI: [-1.43, -0.07]), and without funding support (dSMD: -0.22; 95% CI: [-0.41, -0.02]) tended to show larger effect estimates. This study provides empirical evidence for the development of a specific critical appraisal tool for risk of bias assessments on CHM RCTs.