Facial Expression Recognition for Probing Students’ Emotional Engagement in Science Learning

Abstract Student engagement in science classroom is an essential element for delivering effective instruction. However, the popular method for measuring students’ emotional learning engagement (ELE) relies on self-reporting, which has been criticized for possible bias and lacking fine-grained time solution needed to track the effects of short-term learning interactions. Recent research suggests that students’ facial expressions may serve as an external representation of their emotions in learning. Accordingly, this study proposes a machine learning method to efficiently measure students’ ELE in real classroom. Specifically, a facial expression recognition system based on a multiscale perception network (MP-FERS) was developed by combining the pleasure-displeasure, arousal-nonarousal, and dominance-submissiveness (PAD) emotion models. Data were collected from videos of six physics lessons with 108 students. Meanwhile, students’ academic records and self-reported learning engagement were also collected. The results show that students’ ELE measured by MP-FERS was a significant predictor of academic achievement and a better indicator of true learning status than self-reported ELE. Furthermore, MP-FERS can provide fine-grained time resolution on tracking the changes in students’ ELE in response to different teaching environments such as teacher-centered or student-centered classroom activities. The results of this study demonstrate the validity and utility of MP-FERS in studying students’ emotional learning engagement.