The use of weighted averages of Hedges' d in meta‐analysis: Is it worth it?

Abstract Hedges' d is a measure of effect size widely used to standardize results across different studies in ecological and evolutionary meta‐analyses. When summarizing Hedges' d or other effect sizes across multiple studies, it is considered a best practice to use weighted averages, with weights inversely proportional to the variances of the estimated effect sizes. Importantly, the within‐study variance of Hedges' , , is a function of sample size, , and , the effect size itself. Since true effect sizes are unknown, is also unknown and needs to be estimated, for which numerous approaches have been proposed. We examined the behaviour of inverse‐variance weights, specifically the performance of 14 different weighted and unweighted variants of the Hedges' estimator, to test the conditions under which weighting improved performance. We found that when sample sizes (per group) were >10, there was little difference between the estimators and weighting did not notably improve performance; bias was <5% and differences in root mean squared error were <10% for all estimators, except when the between‐study variance for effect size, , was relatively small compared to the within‐study variance (). We also propose a new estimator that was found to be (a) the most efficient of a subset of minimally biased estimators and (b) had the least bias when estimating τ 2 . Contrary to current guidance, we contend that the simpler unweighted Hedges' d estimator is generally acceptable. Exceptions are when the between‐study variance is small relative to within‐study variance () and there is a large disparity (>3X) in sample sizes for a large proportion of effect sizes in the meta‐analysis. Using unweighted averages may enable more robust inferences by allowing studies for which variance cannot be extracted (a common practical limitation) to be included. That said, weighted averaging is necessary where estimation of is itself of interest.