Investigating the Use of AI-Generated Exercises for Beginner and Intermediate Programming Courses: A ChatGPT Case Study

In recent years, artificial intelligence (AI) has been increasingly used in education and supports teachers in creating educational material and students in their learning progress. AI-driven learning support has recently been further strengthened by the release of ChatGPT, in which users can retrieve explanations for various concepts in a few minutes through chat. However, to what extent the use of AI models, such as ChatGPT, is suitable for the creation of didactically and content-wise good exercises for programming courses is not yet known. Therefore, in this paper, we investigate the use of AI-generated exercises for beginner and intermediate programming courses in higher education using ChatGPT. We created 12 exercise sheets with ChatGPT for a beginner to intermediate programming course focusing on the objects-first approach. We report our process, prompts, and experience using ChatGPT for this task and outline good practices we identified. The generated exercises are assessed and revised, primarily using ChatGPT, until they met the requirements of the programming course. We assessed the quality of these exercises by using them in our external teaching assignment course at the University of Education Ludwigsburg and let the students evaluate them. Results indicate the quality of the generated exercises and the time-saving for creating them using ChatGPT. However, our experience showed that while it is fast to generate a good version of an exercise, almost every exercise requires minor manual changes to improve its quality.