Evaluative Engagement of Human-AI Collaboration in Engineering Problem Solving

Generative artificial intelligence (GAI) is increasingly integrated into Technology-Enhanced STEM Education (TeSTEM), enabling learners to interact with GAI during complex problem solving. While prior research has emphasised prompt formulation, less is known about the role of learners' evaluative engagement in regulating GAI-generated outputs. This study examines the association between evaluative engagement and performance in GAI-supported engineering problem solving. Twenty-eight undergraduate engineering students completed a structured design task using a GAI assistant. Process data, including interaction logs, screen recordings, and simulation artefacts, were analysed alongside instructor performance evaluations. Results indicate that evaluative engagement, reflected in students' ability to detect and respond to GAI-generated errors, exhibited a strong positive association with performance, exceeding the moderate association observed for prompt clarity. These findings highlight evaluative engagement as a central mechanism in effective human-AI collaboration and provide empirical support for the concept of human-AI shared regulation of learning.