Investigating the Construct of Topical Knowledge in Second Language Assessment: A Scenario-Based Assessment Approach

To effectively achieve communicative goals, second language (L2) learners need not only the necessary L2 knowledge, skills, and abilities, but also the relevant topical knowledge. However, in L2 assessment contexts, very few existing tests-in-operation have examined L2 learners' topical knowledge as part of their language performance. This study investigates the construct of topical knowledge in a scenario-based language assessment (SBLA) that simulates real-life language use of building and sharing knowledge. 41 L2 learners at the intermediate to advanced levels from an adult ESL program participated in the study. Topical knowledge was operationalized as content knowledge and lexical knowledge associated with the theme of the SBLA. Descriptive statistics, correlations, Rasch analysis, and repeated measures ANOVAs were used to examine the construct validity of topical knowledge in the SBLA. The results show that L2 learners' topical knowledge encompasses both content and lexical knowledge, but they function differently in the process of building and sharing knowledge. Nevertheless, both play a role in the performance outcomes and should be recognized as an integral component of L2 proficiency. The findings also lend evidentiary support for the use of a highly-contextualized SBLA to broaden our understanding of the underlying construct of L2 communicative ability.