Curriculum-Aligned Conversational AI to Support Pathology Self-Directed Learning in Preclinical Medical Students: A Crossover Randomized Pilot Study (Preprint)

BACKGROUND: AI is reshaping the landscape of medical education, yet the widespread adoption of general-purpose AI tools in student learning raises substantial concerns about curricular alignment, information accuracy, and the absence of pedagogical scaffolding. OBJECTIVE: This crossover pilot study evaluated a curriculum-aligned custom GPT that was developed to support AI-scaffolded self-directed learning (SDL), preparation that remains student-initiated but is supported by structured Socratic prompting, compared with fully unguided traditional preparation in second-year medical students at the University of South Carolina, Floyd School of Medicine (USC Floyd SoM). METHODS: Forty-four second-year medical students voluntarily participated in the 2 faculty-led pathology small group discussions (Cushing disease and prostate cancer), using the custom GPT for one case and conventional resources for the other. Faculty assessed SDL documentation forms and small group discussion performance using standardized rubrics. RESULTS: Students using the curriculum-aligned custom GPT achieved significantly higher SDL rubric scores (10.3 vs 9.4 out of 12; mean difference 0.9, 95% CI 0.5-1.3; P<.001; Cohen d=0.71) and small group discussion scores (14.9 vs 13.7 out of 16; mean difference 1.2, 95% CI 0.7-1.7; P<.001; Cohen d=0.78), with moderate-to-large effect sizes. Delayed retention (end-of-block examination items) did not differ significantly (Wilcoxon signed-rank P=.052). Students also reported significantly greater gains in understanding and pathophysiological reasoning. A total of 34 out of 44 (77%) reported improved reasoning clarity, and 31 out of 44 (70%) improved conceptual understanding. Assessment of 5 prespecified AI-related concern domains did not identify major concerns. CONCLUSIONS: In this crossover pilot study, use of a curriculum-aligned custom GPT was associated with higher faculty-assessed SDL documentation and small group discussion performance. Delayed knowledge retention did not differ significantly in the primary analysis. These findings support further investigation of curriculum-aligned AI tools in larger, multisite studies.