A Preliminary Study on SEPT Speaking Test Design

Generalizability Theory was applied to estimate generalizability coefficients of SEPT Speaking Test with a two-facet crossed design (p × i × r). The G-study and D-study results yielded a moderate but acceptable generalizability coefficient (Eρ 2 = 0.698) of the test and suggested that the assessment made by the raters on all subjects' performance were highly consistent. The subjects performed extremely different on different testing items. Adjustment of rater levels hardly affects the generalizability coefficient thereof, while generalizability coefficient would be greatly lowered if numbers of testing items reduced. It may be accounted for by the fact that individual testing item involves a different area of Ocean Shipping in which individual subject's qualification is different. Therefore, the optimization of the design of SEPT Speaking Test may be achieved by adjusting the sample capacity of the testing items.