Can AI help teachers? Experimental evidence on the effects of generative AI use and AI training on teachers’ lesson preparation

Teachers face considerable workload outside the classroom. Generative Artificial Intelligence (GenAI) could support teachers by enhancing lesson preparation while decreasing workload. Moreover, training on GenAI could further enhance its effectiveness and is increasingly demanded by practitioners. The present study examines whether GenAI (ChatGPT) effectively supports teachers during lesson preparation, and whether participation in AI training improves outcomes. For this purpose, we conducted a preregistered, RCT with N = 204 teachers who were randomly assigned to one of three groups: (1) No access to ChatGPT, (2) access to ChatGPT, and (3) access to ChatGPT with prior AI training. Teachers completed four incentivized lesson preparation tasks: Crafting an introductory quiz, writing a course mail, extending a textbook exercise and creating text and transfer questions. Dependent variables were (a) quality of the prepared materials, assessed by blinded expert ratings based on weighted criteria and (b) time spent on total preparation. Results indicated that GenAI effectively helped teachers during lesson preparation tasks, demonstrated by improved lesson preparation quality and a 34 % reduction in preparation time. AI training did not further improve lesson preparation quality but led to an additional 21 % reduction in preparation time. As revealed by prompt data, AI training led to a distinct prompt strategy (as intended): Teachers with prior training wrote longer and more detailed prompts, that needed less refinement afterwards. Untrained teachers used shorter prompts that required more subsequent prompt iterations. Although the first strategy required less preparation time, both strategies led to equal output quality.