MetaForest: Exploring heterogeneity in meta-analysis using random forests

Meta-analyses in psychology often lack the power to adequately account for between-studies heterogeneity. The number of studies on any topic is typically low, because research is cost- and time-intensive. At the same time, a host of potential moderators are introduced when similar research questions are examined in different labs, sampling from different populations, using idiosyncratic methods and instrumentation. Such between-studies heterogeneity presents a substantial challenge to data aggregation in classic meta-analysis. When the causes for heterogeneity are known a-priori, they can be accounted for using meta-regression. What is currently lacking, however, is an exploratory approach, to be used when heterogeneity is suspected, but it is not known which moderators most strongly influence the observed effect size. Recently, weighted regression trees have been used to explore heterogeneity in meta-analysis. Although this provides a promising first step, single trees have many limitations, which can be overcome by using random forests: A powerful learning algorithm, which is flexible, yet relatively robust to overfitting. The present paper introduces MetaForest: An adaptation of random forests for meta-analysis. We present two simulation studies, which illustrate that, in datasets as small as 20 cases, MetaForest outperforms single trees, in terms of three metrics: 1) Predictive performance; 2) power, as evidenced by the proportion of datasets in which the algorithm achieved a positive R2cv; and 3) the ability to distinguish relevant moderators from irrelevant moderators, using variable importance measures. We discuss how MetaForest can enhance the exploration of between-studies heterogeneity when conducting meta-analyses in diverse bodies of literature.