Multiple Comparisons: Comparisonwise Versus Experimentwise Type I Error Rates and Their Relationship to Power

Some statisticians contend that the experimentwise Type I error rate is the most important attribute of multiple comparison procedures to be used for making all possible pairwise comparisons among treatment means after an analysis of variance. That contention is challenged here. The importance of Type I errors is discussed as well as the occurrence of Type I errors in biological experiments. Also considered is the effect of Type I error protection on power. The two methods of measuring Type I error rate, comparisonwise and experimentwise, are explained, and the reader may decide which kind he wishes to control. Several references cited support Fisher's least significant difference and Duncan's new multiple range test despite their higher-than-nominal experimentwise Type I error rates. These procedures control the comparisonwise Type I error rates and are considerably more powerful in finding differences among treatments than procedures that control the experimentwise Type I error rates.