Engaging Multimodal Literacy Through Generative Artificial Intelligence: A Case Study From an English as Second Language Writing Classroom

ABSTRACT The rise of generative artificial intelligence (GenAI) has provided new space for teachers and scholars to explore how to engage second language writers in new ways of using multimodal resources. However, most of the current research explores the pedagogical potential of GenAI in multimodal writing from a theoretical perspective, while empirical studies on its practical application in English as a second language (ESL) classrooms are still limited. Grounded in the Systemic Functional Linguistics framework, this study investigates how two ESL students focus on different metafunctions in AI‐assisted multimodal writing and how they self‐report increased awareness of multimodality. The findings suggest that the two focal students developed different understandings about the role of images in multimodal storytelling, which potentially leads to one student (Keshav) focusing primarily on ideational meanings while the other student (Yang) paying attention to both ideational and interpersonal meanings. Textual meanings, however, were largely overlooked in both students’ photo essays. This study provides empirical evidence into the use of GenAI in multimodal writing by ESL students and offers valuable insights for designing multimodal writing tasks and suggestions for future research on GenAI and multimodal literacy.