Type I Error Rates and Statistical Power for the James Second-Order Test and the UnivariateFTest in Two-Way Fixed-Effects ANOVA Models Under Heteroscedasticity and/or Nonnormality

Type I error rates and statistical power for the univariate F test and the James second-order test for comparing population means were estimated for the two-factor fixed-effects completely randomized design. When variances were heterogeneous, the univariate F test was shown to have actual Type I error rates greater than the nominal value even when sample sizes were equal. The James second-order test controlled the Type I error rate under heteroscedasticity when population distributions were normal or platykurtic. However, the James second-order test had increased the Type I error rates when the population distributions were skewed. When variances were homogeneous and distributions were normal, the univariate F test and the James second-order test had similar statistical power for testing main effects, but large power differences favoring the univariate F were found for testing interaction effects.