The effectiveness of ChatGPT in assisting high school students in programming learning: evidence from a quasi-experimental research

Programming education gains importance in high schools as the digital age progresses. However, the openness and adaptability of programming languages present unique challenges for instructional practices compared to other subjects. While traditional instructional tools offer limited support, ChatGPT, a groundbreaking Generative Artificial Intelligence, has shown impressive capabilities in natural language processing and knowledge generation. This study explored whether ChatGPT can transcend existing limitations and improve programming education through a quasi-experimental approach with post-hoc interviews in high school classrooms. A total of 153 students participated, and the results from MANCOVA and ANCOVA analyses revealed that students using ChatGPT reported lower levels of flow experience, self-efficacy, and learning achievement compared to those utilizing conventional methods. Post-hoc interviews further revealed that students felt ChatGPT's effectiveness in facilitating their programming learning fell short of their initial expectations. These findings highlight the need to carefully consider the complexity of programming learning tasks and students' cognitive, affective, and interactive dimensions when integrating AI technologies into education. We discuss the implications and provide thorough pedagogical strategies, specifically the guidance–practice–transformation (G–P–T) mode, to maximize the potential of AI tools and support high school programming education, emphasizing the balance of technological innovation with learning best practices.