Pediatric Supracondylar Humerus and Diaphyseal Femur Fractures: A Comparative Analysis of Chat Generative Pretrained Transformer and Google Gemini Recommendations Versus American Academy of Orthopaedic Surgeons Clinical Practice Guidelines

医学 骨科手术 物理疗法 外科
作者
Patrick P. Nian,Amith Umesh,Shae K. Simpson,Olivia C. Tracey,Erikson Nichols,Stephanie L. Logterman,Shevaun M. Doyle,Jessica H. Heyer
出处
期刊:Journal of Pediatric Orthopaedics [Lippincott Williams & Wilkins]
卷期号:45 (4): e338-e344 被引量:10
标识
DOI:10.1097/bpo.0000000000002890
摘要

OBJECTIVE: Artificial intelligence (AI) chatbots, including chat generative pretrained transformer (ChatGPT) and Google Gemini, have significantly increased access to medical information. However, in pediatric orthopaedics, no study has evaluated the accuracy of AI chatbots compared with evidence-based recommendations, including the American Academy of Orthopaedic Surgeons clinical practice guidelines (AAOS CPGs). The aims of this study were to compare responses by ChatGPT-4.0, ChatGPT-3.5, and Google Gemini with AAOS CPG recommendations on pediatric supracondylar humerus and diaphyseal femur fractures regarding accuracy, supplementary and incomplete response patterns, and readability. METHODS: ChatGPT-4.0, ChatGPT-3.5, and Google Gemini were prompted by questions created from 13 evidence-based recommendations (6 from the 2011 AAOS CPG on pediatric supracondylar humerus fractures; 7 from the 2020 AAOS CPG on pediatric diaphyseal femur fractures). Responses were anonymized and independently evaluated by 2 pediatric orthopaedic attending surgeons. Supplementary responses were, in addition, evaluated on whether no, some, or many modifications were necessary. Readability metrics (response length, Flesch-Kincaid reading level, Flesch Reading Ease, Gunning Fog Index) were compared. Cohen Kappa interrater reliability (κ) was calculated. χ 2 analyses and single-factor analysis of variance were utilized to compare categorical and continuous variables, respectively. Statistical significance was set with P <0.05. RESULTS: ChatGPT-4.0, ChatGPT-3.5, and Google Gemini were accurate in 11/13, 9/13, and 11/13, supplementary in 13/13, 11/13, and 13/13, and incomplete in 3/13, 4/13, and 4/13 recommendations, respectively. Of 37 supplementary responses, 17 (45.9%), 19 (51.4%), and 1 (2.7%) required no, some, and many modifications, respectively. There were no significant differences in accuracy ( P = 0.533), supplementary responses ( P = 0.121), necessary modifications ( P = 0.580), and incomplete responses ( P = 0.881). Overall κ was moderate at 0.55. ChatGPT-3.5 provided shorter responses ( P = 0.002), but Google Gemini was more readable in terms of Flesch-Kincaid Grade Level ( P = 0.002), Flesch Reading Ease ( P < 0.001), and Gunning Fog Index ( P = 0.021). CONCLUSIONS: While AI chatbots provided responses with reasonable accuracy, most supplemental information required modification and had complex readability. Improvements are necessary before AI chatbots can be reliably used for patient education. LEVEL OF EVIDENCE: Level IV.

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
刚刚
刚刚
1秒前
smh发布了新的文献求助10
1秒前
Criminology34的应助被Huimin采纳,获得20
1秒前
Criminology34的应助被Huimin采纳,获得10
1秒前
JamesPei的应助被lingzi1015采纳,获得20
1秒前
TCcc发布了新的文献求助10
2秒前
无极微光的应助被Yy采纳,获得20
2秒前
lull完成签到 ,获得积分10
2秒前
2秒前
领导范儿的应助被武科大采纳,获得10
2秒前
八爷啦啦啦完成签到,获得积分10
3秒前
3秒前
3秒前
跳跃的静曼完成签到,获得积分20
3秒前
Ruolin完成签到,获得积分10
3秒前
勤劳的潇完成签到,获得积分10
3秒前
3秒前
慕辰完成签到,获得积分20
3秒前
3秒前
4秒前
4秒前
4秒前
4秒前
传奇3的应助被Maestro_S采纳,获得50
4秒前
无花果的应助被lyn123采纳,获得10
4秒前
仁爱的沁发布了新的文献求助50
5秒前
5秒前
shichao发布了新的文献求助10
5秒前
5秒前
C1完成签到,获得积分10
5秒前
5秒前
亚尔驳回了Hello的应助
5秒前
大模型的应助被Rommel采纳,获得10
5秒前
hqf802802完成签到,获得积分10
5秒前
从容盼旋发布了新的文献求助10
6秒前
Lmx发布了新的文献求助10
6秒前
Colin完成签到,获得积分10
6秒前
6秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1500
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
John McDowells Bildungstheorie 800
八旗制度の研究 600
Analyses de reconnaissances de dettes passées devant les échevins d'Ypres (1249-1291) 600
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8160268
求助须知:如何正确求助?哪些是违规求助? 9555328
关于积分的说明 21727231
捐赠科研通 7750062
什么是DOI,文献DOI怎么找? 3427353
关于科研通互助平台的介绍 2552391
邀请新用户注册赠送积分活动 2471707