Uncertainty in Treatment Rankings: Reanalysis of Network Meta-analyses of Randomized Trials

BACKGROUND: Ranking of interventions is one of the most appealing elements of network meta-analysis. There is, however, little evidence about the reliability of these rankings. PURPOSE: To empirically evaluate the extent of uncertainty in intervention rankings from network meta-analysis. DATA SOURCES: Two previous systematic reviews that involved searches of the Cochrane Library, MEDLINE, and Embase up to July 2012 for articles that included networks of at least 3 interventions. STUDY SELECTION: 58 network meta-analyses involving 1308 randomized trials and 404 interventions with available aggregated outcome data. DATA ANALYSIS: Each network was analyzed with a Bayesian approach. For each intervention, the surface under the cumulative ranking curve (SUCRA) and its 95% credible interval (95% CrI) were estimated. Through use of the SUCRA values, the interventions were then rank-ordered between 0% (worst) and 100% (best). DATA SYNTHESIS: The median width of the 95% CrIs of the SUCRA was 65% (first to third quartile, 38% to 80%). In 28% of networks, there was a 50% or greater probability that the best-ranked treatment was actually not the best. No evidence showed a difference between the best-ranked intervention and the second and third best-ranked interventions in 90% and 71% of comparisons, respectively. In 39 networks with 6 or more interventions, the median probability that 1 of the top 2 interventions was among the bottom 2 was 35% (first to third quartile, 14% to 59%). LIMITATION: This analysis did not consider such factors as the risk of bias within trials or small-study effects that may affect the reliability of rankings. CONCLUSION: Treatment rankings derived from network meta-analyses have a substantial degree of imprecision. Authors and readers should interpret such rankings with great caution. PRIMARY FUNDING SOURCE: Cochrane France.