AI versus human-generated voices and avatars: rethinking user engagement and cognitive load

Abstract As AI-generated content becomes increasingly prevalent in educational settings, understanding its impact on students remains an urgent yet underexplored issue. Although recent studies have begun to explore AI-generated videos, few have specifically examined the pedagogical potential of combining AI voices and avatars. This study investigates the differences in user engagement and cognitive load induced by AI-generated short videos compared to human-based ones in the context of foreign language learning. Using a 2 × 2 experimental design, the study explored four combinations: AI voice & AI avatar, AI voice & Human avatar, Human voice & AI avatar, and Human voice & Human avatar. Results from linear mixed models and Fuzzy C-Means clustering revealed that while both AI voice and AI avatar can improve user engagement independently, a significant improvement in engagement is only observed when both voice and avatar are AI-generated. Similarly, while AI voice and AI avatar can reduce extraneous cognitive load independently, a significant reduction in extraneous cognitive load is only observed when both voice and avatar are AI-generated. This study offers valuable insights for integrating AI-generated short videos into foreign language learning and the broader educational landscape, paving the way for AI-generated content and innovative approaches to multimedia learning.