已入深夜,您辛苦了!由于当前在线用户较少,发布求助请尽量完整地填写文献信息,科研通机器人24小时在线,伴您度过漫漫科研夜!祝你早点完成任务,早点休息,好梦!

Optimizing ChatGPT’s Interpretation and Reporting of Delirium Assessment Outcomes: Exploratory Study

口译(哲学) 谵妄 探索性研究 心理学 医学 计算机科学 精神科 社会学 社会科学 程序设计语言
作者
Yong Kyung Choi,Shih‐Yin Lin,Donna Marie Fick,Richard W. Shulman,Sangil Lee,Priyanka Shrestha,K. P. Santoso
出处
期刊:JMIR formative research [JMIR Publications]
卷期号:8: e51383-e51383 被引量:11
标识
DOI:10.2196/51383
摘要

BACKGROUND: Generative artificial intelligence (AI) and large language models, such as OpenAI's ChatGPT, have shown promising potential in supporting medical education and clinical decision-making, given their vast knowledge base and natural language processing capabilities. As a general purpose AI system, ChatGPT can complete a wide range of tasks, including differential diagnosis without additional training. However, the specific application of ChatGPT in learning and applying a series of specialized, context-specific tasks mimicking the workflow of a human assessor, such as administering a standardized assessment questionnaire, followed by inputting assessment results in a standardized form, and interpretating assessment results strictly following credible, published scoring criteria, have not been thoroughly studied. OBJECTIVE: This exploratory study aims to evaluate and optimize ChatGPT's capabilities in administering and interpreting the Sour Seven Questionnaire, an informant-based delirium assessment tool. Specifically, the objectives were to train ChatGPT-3.5 and ChatGPT-4 to understand and correctly apply the Sour Seven Questionnaire to clinical vignettes using prompt engineering, assess the performance of these AI models in identifying and scoring delirium symptoms against scores from human experts, and refine and enhance the models' interpretation and reporting accuracy through iterative prompt optimization. METHODS: We used prompt engineering to train ChatGPT-3.5 and ChatGPT-4 models on the Sour Seven Questionnaire, a tool for assessing delirium through caregiver input. Prompt engineering is a methodology used to enhance the AI's processing of inputs by meticulously structuring the prompts to improve accuracy and consistency in outputs. In this study, prompt engineering involved creating specific, structured commands that guided the AI models in understanding and applying the assessment tool's criteria accurately to clinical vignettes. This approach also included designing prompts to explicitly instruct the AI on how to format its responses, ensuring they were consistent with clinical documentation standards. RESULTS: Both ChatGPT models demonstrated promising proficiency in applying the Sour Seven Questionnaire to the vignettes, despite initial inconsistencies and errors. Performance notably improved through iterative prompt engineering, enhancing the models' capacity to detect delirium symptoms and assign scores. Prompt optimizations included adjusting the scoring methodology to accept only definitive "Yes" or "No" responses, revising the evaluation prompt to mandate responses in a tabular format, and guiding the models to adhere to the 2 recommended actions specified in the Sour Seven Questionnaire. CONCLUSIONS: Our findings provide preliminary evidence supporting the potential utility of AI models such as ChatGPT in administering standardized clinical assessment tools. The results highlight the significance of context-specific training and prompt engineering in harnessing the full potential of these AI models for health care applications. Despite the encouraging results, broader generalizability and further validation in real-world settings warrant additional research.

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
科研通AI6.2的应助被认真的莹采纳,获得10
3秒前
小小瓜发布了新的文献求助10
3秒前
3秒前
4秒前
调皮访烟完成签到,获得积分20
4秒前
十九完成签到,获得积分10
4秒前
6秒前
调皮访烟发布了新的文献求助10
7秒前
7秒前
银杏发布了新的文献求助10
10秒前
Cynthia完成签到 ,获得积分10
10秒前
VDC的应助被科研通管家采纳,获得30
11秒前
11秒前
11秒前
天天快乐的应助被科研通管家采纳,获得10
11秒前
嘻嘻哈哈的应助被科研通管家采纳,获得10
11秒前
12秒前
12秒前
今后的应助被rebecca采纳,获得10
12秒前
Cakoibao的应助被科研通管家采纳,获得10
12秒前
海伦发布了新的文献求助10
12秒前
嘻嘻哈哈的应助被科研通管家采纳,获得10
12秒前
科研通AI2S的应助被王百万采纳,获得10
16秒前
冰冰完成签到 ,获得积分10
16秒前
周周发布了新的文献求助10
16秒前
宋忠赫完成签到,获得积分10
16秒前
Thaumatin完成签到,获得积分20
16秒前
科研通AI6.4的应助被HL采纳,获得10
17秒前
zhouzhou发布了新的文献求助10
17秒前
小小瓜完成签到,获得积分10
18秒前
天天向上的应助被DawnQinghui采纳,获得10
19秒前
23秒前
23秒前
23秒前
26秒前
Akim的应助被奔跑的猫猫采纳,获得10
26秒前
ket的应助被奔跑的猫猫采纳,获得10
26秒前
26秒前
呆萌威完成签到,获得积分10
27秒前
28秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 2000
Handbook of Sport Coaching Research 1000
Orthognathic Surgery - 2 Volume Set 500
高分子基功能界面水蒸发材料与应用研究 500
外场增强的水汽发电研究 500
基于结构调控的功能化水诱导发电材料与器件研究 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8194612
求助须知:如何正确求助?哪些是违规求助? 9580394
关于积分的说明 21870930
捐赠科研通 7794182
什么是DOI,文献DOI怎么找? 3437711
关于科研通互助平台的介绍 2561001
邀请新用户注册赠送积分活动 2484411