Unpacking individual differences in AI teachable agent engagement: Associations with engagement and mathematics exam performance across learner groups

The study investigates individual differences in middle school math learning during interactions with an AI teachable agent, focusing on how students' individual characteristics and their use of tutoring strategies shape learning outcomes. Data analysis from 192 students in the experimental group and 205 students in the business-as-usual instruction group indicated that study condition, baseline math achievement, race, and literacy significantly predicted performance on a high-stakes math assessment; strength of association between tutoring strategy use and learning performance varied across student user groups. Lag sequential analysis further revealed (a) more questioning and sending off-topic messages consecutively among male learners, (b) more directional strategy sequence from feedback to self-reflection for learners in the represented racial group, and (c) more adaptive tutoring strategy use among learners of average-and-above English literacy. These findings highlight the importance of considering learner characteristics and engagement patterns when designing adaptive AI-supported tutoring systems. Educational relevance statement This study investigates middle-school students' individual differences when interacting with an AI-powered teachable agent for math learning. Analysis of engagement data and high-stakes state exam scores from historically underrepresented learners revealed (a) an overall effectiveness of engaging in learning-by-teaching with the teachable agent, (b) differential associations between of tutoring strategy use and math learning outcomes across user groups, and (c) distinct behavioral patterns across race, gender, and English literacy. The findings highlighted the effectiveness of the AI teachable agent and that integrating teachable agents into math learning should not be treated as a one-size-fits-all approach and underscored the importance of designing student-agent interactions that account for learner differences to support optimal engagement and learning outcomes.