Engagement patterns of middle school students with AI teachable agents in mathematics learning

This study investigates how secondary students engage with an AI teachable agent (TA) during mathematics learning, with particular focus on learners whose performance declined after interacting with the TA system. Using a mixed-methods design, we analyzed dialogue logs from two subgroups: a Declined Group (DG; n = 206), whose post-test scores decreased, and an Improved Group (IG; n = 327), whose scores increased. Analyses examined interaction modes and behavioral, emotional, and cognitive engagement. Passive interaction was most prevalent in DG (36 %), whereas IG more frequently demonstrated constructive interaction (62.78 %). DG exhibited high variability in behavioral engagement: although they completed more sessions and generated more utterances per session, their completion rate (0.35) was lower than that of IG (0.58). Regarding emotional engagement, boredom was the most frequent non-neutral emotion in DG (50.4 %) and tended to rise as sessions progressed, whereas IG expressed more positive (30 %) than negative emotions (17.94 %). For cognitive engagement, most students displayed surface-level knowledge acquisition with limited application to novel or complex tasks. Notably, within DG, greater behavioral activity and positive emotions were sometimes associated with lower learning gains, often when such activity reflected off-task dialogue or superficial goal completion. These findings highlight classroom challenges in AI-supported learning and suggest design implications for TAs that scaffold proactive interaction, detect emerging boredom, and redirect high-volume yet low-yield behaviors toward meaningful engagement.