Performance of ChatGPT in the In-Training Examination for Anesthesiology and Pain Medicine Residents in South Korea: Observational Study

麻醉学 观察研究 止痛药 医学 培训(气象学) 物理疗法 家庭医学 内科学 麻醉 地理 气象学
作者
Soo‐Hyuk Yoon,Seok Kyeong Oh,Byung Gun Lim,Ho‐Jin Lee
出处
期刊:JMIR medical education [JMIR Publications]
卷期号:10: e56859-e56859
标识
DOI:10.2196/56859
摘要

Background ChatGPT has been tested in health care, including the US Medical Licensing Examination and specialty exams, showing near-passing results. Its performance in the field of anesthesiology has been assessed using English board examination questions; however, its effectiveness in Korea remains unexplored. Objective This study investigated the problem-solving performance of ChatGPT in the fields of anesthesiology and pain medicine in the Korean language context, highlighted advancements in artificial intelligence (AI), and explored its potential applications in medical education. Methods We investigated the performance (number of correct answers/number of questions) of GPT-4, GPT-3.5, and CLOVA X in the fields of anesthesiology and pain medicine, using in-training examinations that have been administered to Korean anesthesiology residents over the past 5 years, with an annual composition of 100 questions. Questions containing images, diagrams, or photographs were excluded from the analysis. Furthermore, to assess the performance differences of the GPT across different languages, we conducted a comparative analysis of the GPT-4’s problem-solving proficiency using both the original Korean texts and their English translations. Results A total of 398 questions were analyzed. GPT-4 (67.8%) demonstrated a significantly better overall performance than GPT-3.5 (37.2%) and CLOVA-X (36.7%). However, GPT-3.5 and CLOVA X did not show significant differences in their overall performance. Additionally, the GPT-4 showed superior performance on questions translated into English, indicating a language processing discrepancy (English: 75.4% vs Korean: 67.8%; difference 7.5%; 95% CI 3.1%-11.9%; P=.001). Conclusions This study underscores the potential of AI tools, such as ChatGPT, in medical education and practice but emphasizes the need for cautious application and further refinement, especially in non-English medical contexts. The findings suggest that although AI advancements are promising, they require careful evaluation and development to ensure acceptable performance across diverse linguistic and professional settings.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
jenny完成签到,获得积分10
2秒前
王婧jjj发布了新的文献求助10
3秒前
乐观红牛发布了新的文献求助10
3秒前
TN完成签到 ,获得积分10
4秒前
顾矜应助俭朴夜香采纳,获得10
11秒前
12秒前
kytzh完成签到,获得积分10
13秒前
彭于晏应助Lqiang采纳,获得10
14秒前
15秒前
英俊的铭应助103921wjk采纳,获得10
15秒前
16秒前
16秒前
科研通AI5应助刀客特幽采纳,获得10
17秒前
pluto应助博士采纳,获得10
18秒前
归尘发布了新的文献求助10
20秒前
123457关注了科研通微信公众号
20秒前
儒雅的焦完成签到 ,获得积分10
20秒前
20秒前
20秒前
太空工程师完成签到,获得积分10
21秒前
Nia发布了新的文献求助10
23秒前
努力成为科研大佬完成签到,获得积分10
24秒前
103921wjk发布了新的文献求助10
25秒前
里海鱼完成签到,获得积分10
26秒前
26秒前
szh123完成签到 ,获得积分10
30秒前
小白应助kyJYbs采纳,获得20
31秒前
NexusExplorer应助yue采纳,获得10
32秒前
33秒前
36秒前
科研通AI5应助刀客特幽采纳,获得10
37秒前
37秒前
37秒前
平常剑鬼发布了新的文献求助10
39秒前
LQX2141完成签到 ,获得积分10
41秒前
echooooo发布了新的文献求助10
42秒前
Lqiang发布了新的文献求助10
43秒前
43秒前
43秒前
pluto应助SYY采纳,获得20
43秒前
高分求助中
【此为提示信息,请勿应助】请按要求发布求助,避免被关 20000
Encyclopedia of Geology (2nd Edition) 2000
Maneuvering of a Damaged Navy Combatant 650
Периодизация спортивной тренировки. Общая теория и её практическое применение 310
Mixing the elements of mass customisation 300
the MD Anderson Surgical Oncology Manual, Seventh Edition 300
Nucleophilic substitution in azasydnone-modified dinitroanisoles 300
热门求助领域 (近24小时)
化学 材料科学 医学 生物 工程类 有机化学 物理 生物化学 纳米技术 计算机科学 化学工程 内科学 复合材料 物理化学 电极 遗传学 量子力学 基因 冶金 催化作用
热门帖子
关注 科研通微信公众号,转发送积分 3780364
求助须知:如何正确求助?哪些是违规求助? 3325704
关于积分的说明 10224008
捐赠科研通 3040823
什么是DOI,文献DOI怎么找? 1669040
邀请新用户注册赠送积分活动 799013
科研通“疑难数据库(出版商)”最低求助积分说明 758648