Using explainable AI to unravel classroom dialogue analysis: Effects of explanations on teachers' trust, technology acceptance and cognitive load

Abstract Deep neural networks are increasingly employed to model classroom dialogue and provide teachers with prompt and valuable feedback on their teaching practices. However, these deep learning models often have intricate structures with numerous unknown parameters, functioning as black boxes. The lack of clear explanations regarding their classroom dialogue analysis likely leads teachers to distrust and underutilize these AI‐powered models. To tackle this issue, we leveraged explainable AI to unravel classroom dialogue analysis and conducted an experiment to evaluate the effects of explanations. Fifty‐nine pre‐service teachers were recruited and randomly assigned to either a treatment ( n = 30) or control ( n = 29) group. Initially, both groups learned to analyse classroom dialogue using AI‐powered models without explanations. Subsequently, the treatment group received both AI analysis and explanations, while the control group continued to receive only AI predictions. The results demonstrated that teachers in the treatment group exhibited significantly higher levels of trust in and technology acceptance of AI‐powered models for classroom dialogue analysis compared to those in the control group. Notably, there were no significant differences in cognitive load between the two groups. Furthermore, teachers in the treatment group expressed high satisfaction with the explanations. During interviews, they also elucidated how the explanations changed their perceptions of model features and attitudes towards the models. This study is among the pioneering works to propose and validate the use of explainable AI to address interpretability challenges within deep learning‐based models in the context of classroom dialogue analysis. Practitioner notes What is already known about this topic Classroom dialogue is recognized as a crucial element in the teaching and learning process. Researchers have increasingly utilized AI techniques, particularly deep learning methods, to analyse classroom dialogue. Deep learning‐based models, characterized by their intricate structures, often function as black boxes, lacking the ability to provide transparent explanations regarding their analysis. This limitation can result in teachers harbouring distrust and underutilizing these models. What this paper adds This paper highlights the importance of incorporating explainable AI approaches to tackle the interpretability issues associated with deep learning‐based models utilized for classroom dialogue analysis. Through an experimental study, this paper demonstrates that providing model explanations enhances teachers' trust in and technology acceptance of AI‐powered classroom dialogue models, without increasing their cognitive load. Teachers express satisfaction with the model explanations provided by explainable AI. Implications for practice and/or policy The integration of explainable AI can effectively address the challenge of interpretability in complex AI‐powered models used for analysing classroom dialogue. Intelligent teaching systems designed for classroom dialogue can benefit from advanced AI models and explainable AI approaches, which offer users both automated analysis and clear explanations. By enabling users to understand the underlying rationale behind the analysis, the explanations can contribute to fostering trust and acceptance of the AI models among users.