A Fail-Safe N for Effect Size in Meta-Analysis

Rosenthan's (1979) concept of fail-safe N has thus far been applied to probability levels exclusively. This note introduces a fail-safe TV for effect size. Rosenthal's (1979) fail-safe N was an ingenious response to the so-called file drawer problem in research integration, in that it provided a direct assessment of the threat posed by sampling bias in the literature search. The algorithm was derived from the Stouffer method (Mosteller & Bush, 1954), in which an overall Z-score is computed by summing individual Z-scores and dividing by the square root of the number of scores. A significant overall Z-score, Rosenthal reasoned, could be made nonsignificant by adding some knowable number of hypothetical studies that averaged null results. The exact number of such studies needed to bring a significant overall p level up to some critical level (e.g., .05) has since become known as the fail-safe N (Nfs). The formula is