A Comparative Evaluation of Psychometric Meta‐Analysis Methods in Management and Applied Psychology: Toward a Nuanced Understanding of Their Accuracy

ABSTRACT This study compares the accuracy of five psychometric meta‐analysis methods—that is, two new explicit/blended random‐effects (RE) methods with RE study weights based on Hedges and Vevea (1998) and Schmidt and Hunter (2015), respectively, two existing RE methods with fixed‐effect (FE) study weights (Raju et al. 2006; Schmidt and Hunter 2015), and a traditional FE method (Hedges and Olkin 1985)—while correcting for sampling error, measurement error, and range restriction using simulated data. We evaluate their absolute and relative accuracy using a series of Monte Carlo simulations with a broad range of realistic conditions. The findings of this study indicate that all RE methods perform well in estimating population mean effect sizes and, to a lesser extent, their heterogeneity (true standard deviation) and precision (95% confidence interval coverage). As expected, each RE method outperforms the FE method. Moreover, the two explicit/blended RE methods with RE study weights are more accurate than the other two RE methods with FE study weights in terms of precision estimation. Overall, the blended RE method with RE weights based on Schmidt and Hunter (2015) appears to be relatively more accurate than the other RE methods, but actual differences are rather small. Notably, the widely used Schmidt and Hunter's method, while generally accurate, is not the most statistically optimal choice. Overall, this study underscores the importance of a nuanced understanding of the strengths and weaknesses of each psychometric meta‐analysis method. We discuss this study's implications for meta‐analysis methods and applications along with study limitations and future research directions.