Empirical comparison of subgroup effects in conventional and individual patient data meta-analyses

OBJECTIVES: Individual patient data (IPD) meta-analyses have been proposed as a major improvement in meta-analytic methods to study subgroup effects. Subgroup effects of conventional and IPD meta-analyses using identical data have not been compared. Our objective is to compare such subgroup effects using the data of six trials (n = 1,643) on the effectiveness of antibiotics in children with acute otitis media (AOM). METHODS: Effects (relative risks, risk differences [RD], and their confidence intervals [CI]) of antibiotics in subgroups of children with AOM resulting from (i) conventional meta-analysis using summary statistics derived from published data (CMA), (ii) two-stage approach to IPD meta-analysis using summary statistics derived from IPD (IPDMA-2), and (iii) one-stage approach to IPD meta-analysis where IPD is pooled into a single data set (IPDMA-1) were compared. RESULTS: In the conventional meta-analysis, only two of the six studies were included, because only these reported on relevant subgroup effects. The conventional meta-analysis showed larger (age < 2 years) or smaller (age > or = 2 years) subgroup effects and wider CIs than both IPD meta-analyses (age < 2 years: RDCMA -21 percent, RDIPDMA-1 -16 percent, RDIPDMA-2 -15 percent; age > or =2 years: RDCMA -5 percent, RDIPDMA-1 -11 percent, RDIPDMA-2 -11 percent). The most important reason for these discrepant results is that the two studies included in the conventional meta-analysis reported outcomes that were different both from each other and from the IPD meta-analyses. CONCLUSIONS: This empirical example shows that conventional meta-analyses do not allow proper subgroup analyses, whereas IPD meta-analyses produce more accurate subgroup effects. We also found no differences between the one- and two-stage meta-analytic approaches.