A Meta‐Analysis of the Impact of Generative Artificial Intelligence on Learning Outcomes

ABSTRACT Background With the rapid advancement of technology, the integration of Generative Artificial Intelligence (GAI) in education has gained considerable attention. Many studies have examined GAI's impact on learning outcomes, yet their conclusions are inconsistent, highlighting the need for a comprehensive review to clarify its overall effects and identify influential factors. Objectives This study aims to conduct a meta‐analysis of the effects of GAI on student learning outcomes across cognitive, competency and affective dimensions. Additionally, it seeks to explore how various moderating factors, including subject discipline, instructional duration, knowledge type, prior knowledge and tool type, influence GAI's effectiveness. Methods A meta‐analysis was performed on 34 experimental and quasi‐experimental studies published internationally. Effect sizes were calculated for overall learning outcomes and categorised by dimension. Further analysis was conducted to assess the influence of moderating variables on the impact of GAI. Results The meta‐analysis indicates that Generative Artificial Intelligence has a significant positive impact on overall learning outcomes, with a combined effect size of 0.68 ( p < 0.001). The impact is particularly pronounced in the cognitive dimension ( g = 0.795) and the competency dimension ( g = 0.711), while its effect on the affective dimension ( g = 0.507) is moderate but still significant. The analysis of moderating variables reveals that the effectiveness of GAI is influenced by discipline type but is not significantly affected by instructional period, knowledge type, prior knowledge level, or tool type. Specifically, GAI exhibits the highest positive effects in mathematics, science and humanities, whereas its impact is relatively lower yet still significant in computer science and medical/nursing education. Additionally, GAI's effectiveness does not significantly differ across various instructional periods, different knowledge types, learners with varying prior knowledge levels, or different AI tool versions. Conclusions To optimise GAI's use in education, the study suggests aligning GAI with specific subject needs, adapting tools for different student levels, integrating GAI with traditional teaching and establishing monitoring mechanisms. These strategies aim to maximise GAI's positive impact on learning efficiency and quality across educational settings.