Effectiveness of generative artificial intelligence-based teaching versus traditional teaching methods in medical education: a meta-analysis of randomized controlled trials

BACKGROUND: Artificial intelligence (AI) has demonstrated remarkable capabilities across diverse medical applications, potentially revolutionizing healthcare delivery systems. This systematic review and meta-analysis investigated the comparative effectiveness of generative artificial intelligence (GAI)-based teaching methodologies versus conventional pedagogical approaches on educational outcomes among medical students. METHODS: We conducted a comprehensive literature search across multiple electronic databases, including PubMed, Cochrane Library, EMBASE, and Web of Science, encompassing studies published from January 2014 through January 2025. The review focused on randomized controlled trials (RCTs) that compared GAI-based teaching interventions with traditional instructional teaching methods in medical students. RESULTS: The meta-analysis incorporated 11 eligible RCTs, comprising 786 medical students. Pooled analysis revealed no statistically significant difference in knowledge acquisition scores between GAI-based and traditional teaching approaches (standardized mean difference [SMD] 0.27, 95% confidence interval [CI] -0.31 to 0.85; p = 0.36). However, subgroup analysis indicated enhanced knowledge performance in the GAI group specifically for extended learning periods (exceeding one week) and practice-oriented courses. GAI-based instruction demonstrated superior outcomes in practical skill development compared to conventional methods (SMD 0.63, 95% CI 0.10-1.16; p = 0.02). Students in the GAI group reported significantly higher satisfaction scores with their learning experience. CONCLUSION: While theoretical knowledge acquisition remains comparable between teaching modalities, the distinctive advantages of GAI-based approaches in practical skill development warrant their integration into medical curricula. Future research should focus on optimizing the integration of GAI-based teaching methods, standardizing implementation protocols, and evaluating long-term educational outcomes. TRIAL REGISTRATION: This protocol was registered on the International Platform of Registered Systematic Review and Meta-analysis Protocols (INPLASY) with the registration number INPLASY202510006. Registered on 2 January 2025.