Evaluating the Effectiveness of Large Language Models in Providing Patient Education for Chinese Patients With Ocular Myasthenia Gravis: Mixed Methods Study

可读性 可用性 医疗保健 医学 重症肌无力 家庭医学 病人教育 复视 医学教育 心理学 计算机科学 外科 内科学 人机交互 经济 经济增长 程序设计语言
作者
Bin Wei,Lili Yao,Xin Hu,Y.C. Hu,Jie Rao,Yu Ji,Zhuoer Dong,Yichong Duan,Xiaorong Wu
出处
期刊:Journal of Medical Internet Research [JMIR Publications]
卷期号:27: e67883-e67883 被引量:9
标识
DOI:10.2196/67883
摘要

BACKGROUND: Ocular myasthenia gravis (OMG) is a neuromuscular disorder primarily affecting the extraocular muscles, leading to ptosis and diplopia. Effective patient education is crucial for disease management; however, in China, limited health care resources often restrict patients' access to personalized medical guidance. Large language models (LLMs) have emerged as potential tools to bridge this gap by providing instant, AI-driven health information. However, their accuracy and readability in educating patients with OMG remain uncertain. OBJECTIVE: The purpose of this study was to systematically evaluate the effectiveness of multiple LLMs in the education of Chinese patients with OMG. Specifically, the validity of these models in answering patients with OMG-related questions was assessed through accuracy, completeness, readability, usefulness, and safety, and patients' ratings of their usability and readability were analyzed. METHODS: The study was conducted in two phases: 130 choice ophthalmology examination questions were input into 5 different LLMs. Their performance was compared with that of undergraduates, master's students, and ophthalmology residents. In addition, 23 common patients with OMG-related patient questions were posed to 4 LLMs, and their responses were evaluated by ophthalmologists across 5 domains. In the second phase, 20 patients with OMG interacted with the 2 LLMs from the first phase, each asking 3 questions. Patients assessed the responses for satisfaction and readability, while ophthalmologists evaluated the responses again using the 5 domains. RESULTS: ChatGPT o1-preview achieved the highest accuracy rate of 73% on 130 ophthalmology examination questions, outperforming other LLMs and professional groups like undergraduates and master's students. For 23 common patients with OMG-related questions, ChatGPT o1-preview scored highest in correctness (4.44), completeness (4.44), helpfulness (4.47), and safety (4.6). GEMINI (Google DeepMind) provided the easiest-to-understand responses in readability assessments, while GPT-4o had the most complex responses, suitable for readers with higher education levels. In the second phase with 20 patients with OMG, ChatGPT o1-preview received higher satisfaction scores than Ernie 3.5 (Baidu; 4.40 vs 3.89, P=.002), although Ernie 3.5's responses were slightly more readable (4.31 vs 4.03, P=.01). CONCLUSIONS: LLMs such as ChatGPT o1-preview may have the potential to enhance patient education. Addressing challenges such as misinformation risk, readability issues, and ethical considerations is crucial for their effective and safe integration into clinical practice.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
刀河王发布了新的文献求助10
刚刚
刚刚
刚刚
cdercder的应助被jun采纳,获得10
刚刚
1秒前
CJB发布了新的文献求助10
1秒前
初景发布了新的文献求助10
1秒前
2秒前
charles发布了新的文献求助10
2秒前
2秒前
feather完成签到,获得积分10
2秒前
2秒前
Crystal完成签到,获得积分10
2秒前
楠湘完成签到,获得积分10
2秒前
2秒前
滴滴完成签到,获得积分10
3秒前
982289172发布了新的文献求助10
3秒前
你好呀完成签到 ,获得积分10
3秒前
starry完成签到,获得积分10
4秒前
沉默诗兰完成签到,获得积分10
4秒前
tong发布了新的文献求助10
4秒前
5秒前
VV发布了新的文献求助10
5秒前
酒酒完成签到,获得积分10
5秒前
Tia完成签到 ,获得积分10
5秒前
6秒前
Dawn完成签到,获得积分10
6秒前
科研通AI6.3的应助被清辉影采纳,获得10
6秒前
在水一方的应助被fortuna采纳,获得10
6秒前
7秒前
7秒前
文艺紫菜发布了新的文献求助10
8秒前
kinsley完成签到,获得积分10
8秒前
8秒前
Lucas的应助被punch采纳,获得10
8秒前
傢誠发布了新的文献求助10
8秒前
8秒前
快乐的鱼发布了新的文献求助10
9秒前
orixero的应助被zhz采纳,获得10
9秒前
9秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
八旗制度の研究 600
高分子基功能界面水蒸发材料与应用研究 500
外场增强的水汽发电研究 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8187592
求助须知:如何正确求助?哪些是违规求助? 9575705
关于积分的说明 21842471
捐赠科研通 7785407
什么是DOI,文献DOI怎么找? 3435057
关于科研通互助平台的介绍 2559461
邀请新用户注册赠送积分活动 2481952