REAL AND NOMINAL SIGNIFICANCE LEVELS IN STATISTICAL HYPOTHESIS TESTING

In the statistical hypothesis testing, critical values often point to a priori fixed (nominal) significance levels. As such, typically researcher uses the values of three numbers 0.01, 0.05, 0.1, to which may be added a few levels: 0.001, 0.005, 0.02, and others. However, for the statistics with discrete distribution functions, which, in particular, include all nonparametric statistical tests, the real significance levels may be different from the nominal, differ at times. Under the real significance level we refer to the highest possible significance level of discrete statistics, not exceeding a given nominal significance level (ie, the transition to the next highest possible value corresponding discrete statistical significance level is greater than a predetermined nominal). In the article, we have discussed the difference between nominal and real significance levels on the example of nonparametric tests for the homogeneity of two independent samples. We have also studied two-sample Wilcoxon test, the criterion of van der Waerden, Smirnov two-sample two-sided test, sign test, runs test (Wolfowitz) and calculated the real significance levels of the criteria for nominal significance level of 0.05. The study of the power of these statistical tests is accomplished by means of Monte Carlo method. The main conclusion: the use of nominal significance levels instead of real significance levels for discrete statistics is inadmissible for small sample sizes