Comparing the Effects of Instructor Manual Feedback and ChatGPT Intelligent Feedback on Collaborative Programming in China's Higher Education

Artificial general intelligence (AGI) has gained increasing global attention as the field of large language models undergoes rapid development. Due to its human-like cognitive abilities, the AGI system has great potential to help instructors provide detailed, comprehensive, and individualized feedback to students throughout the educational process. ChatGPT, as a preliminary version of the AGI system, has the potential to improve programming education. In programming, students often have difficulties in writing codes and debugging errors, whereas ChatGPT can provide intelligent feedback to support students’ programming learning process. This research implemented intelligent feedback generated by ChatGPT to facilitate collaborative programming among student groups and further compared the effects of ChatGPT with instructors’ manual feedback on programming. This research employed a variety of learning analytics methods to analyze students’ computer programming performances, cognitive and regulation discourses, and programming behaviors. Results indicated that no substantial differences were identified in students’ programming knowledge acquisition and group-level programming product quality when both instructor manual feedback and ChatGPT intelligent feedback were provided. ChatGPT intelligent feedback facilitated students’ regulation-oriented collaborative programming, while instructor manual feedback facilitated cognition-oriented collaborative discussions during programming. Compared to the instructor manual feedback, ChatGPT intelligent feedback was perceived by students as having more obvious strengths as well as weaknesses. Drawing from the results, this research offered pedagogical and analytical insights to enhance the integration of ChatGPT into programming education at the higher education context. This research also provided a new perspective on facilitating collaborative learning experiences among students, instructors, and the AGI system.