Comparative Performance of Three GPT Models on Japanese Dental Board-Style Multiple-Choice Questions

Large language models (LLMs) are increasingly used in professional examinations, but their relative performance on dental board-style questions remains unclear. This study compared two reasoning-optimized models, GPT-o3 and GPT-5T, with a general-purpose multimodal model, GPT-4o, using 399 Japanese dental board-style multiple-choice questions from 2018 to 2022. All questions were presented in Japanese, and items originally accompanied by charts, photographs, or other figures were analyzed separately from items without visual materials. Accuracy and item-level agreement were assessed using pairwise McNemar tests, stratified analyses according to the original presence of visual materials, the Breslow–Day test for homogeneity of odds ratios, and two-proportion z-tests. GPT-5T achieved the highest overall accuracy (294/399, 73.7%), followed by GPT-o3 (257/399, 64.4%) and GPT-4o (255/399, 63.9%). Pairwise McNemar tests showed that GPT-5T outperformed both GPT-4o (Holm-adjusted p = 0.00098) and GPT-o3 (Holm-adjusted p = 0.00072), whereas GPT-o3 and GPT-4o did not differ significantly (Holm-adjusted p = 0.920). Accuracy was lower for questions originally containing visual materials than for questions without such materials across all three models (GPT-4o: 49.7% vs. 72.2%; GPT-o3: 55.1% vs. 69.8%; GPT-5T: 59.9% vs. 81.8%). The advantage of GPT-5T was more evident in questions without visual materials, and heterogeneity across question formats was observed for GPT-5T versus GPT-o3. GPT-5T showed the strongest performance in this dataset. Questions originally containing visual materials were associated with lower accuracy across all models. Because the comparison was based on distinct item groups rather than experimentally manipulated visual conditions, this result should be interpreted as a difference across question formats and may also reflect differences in item composition and difficulty between the two groups.