Does Test Preparation Work? Implications for Score Validity

This article reports an empirical study that examined the pattern of test preparation for College English Test Band 4 (CET4) and the differential effects of test preparation practices on its scores, thereby drawing implications for CET4 score validity. Data collection involved 1,003 test takers of CET4. A pretest was administered at the beginning of a 2-month test preparation period; a posttest and a test preparation questionnaire were administered about 9 weeks later. Multiple regression and structural equation modeling were used in data analysis. The study found that test preparation pattern was better explained by the perceived functions of strategy use in improving test scores rather than the information processing mechanism underlying strategy use. Test preparation did improve test scores, but the effects were primarily from preparation practices via narrowing the curriculum, especially drilling. Although the effects were small in absolute terms, they represented almost one third of the effects from the pretest. Findings of this study suggest that the extrapolation validity of CET4 scores may warrant special attention in future validation studies.