主题分析
计算机科学
定性研究
叙述的
背景(考古学)
定性性质
数据科学
可扩展性
推论
自然语言处理
语用学
语篇分析
定性分析
专题地图
规范性
人工智能
知识管理
可视化
心理学
编码(社会科学)
医疗保健
提升(金属加工)
管理科学
钥匙(锁)
工作(物理)
作者
Ranjita Misra,Rajashree Dahal,Brenna Kirk,Raihan Khan,Gizem Dogan,Robin Chataut,Prashnna Gyawali
标识
DOI:10.1177/16094069261426100
摘要
Qualitative data analysis, particularly thematic analysis, is a widely used method for uncovering patterns and insights in narrative data but often faces challenges such as being resource-intensive and susceptible to researcher bias. In recent years, Large Language Models (LLMs) have emerged as promising tools to assist human coders in conducting thematic analysis, offering efficiency and scalability in processing large datasets. However, existing work primarily relies on tools like ChatGPT, raising privacy concerns—especially when analyzing sensitive healthcare data—and lacks systematic validation through comparisons with human coders. This study aims to evaluate the potential of open-source LLMs for qualitative data analysis. Semi-structured patient interviews (n = 34) conducted by a trained qualitative researcher provided the dataset, with traditional qualitative (inductive) analysis by two researchers serving as the gold standard. Two open-source LLMs, Gemma2 and Llama3.1, were used to generate codes, and their performance was assessed based on factors such as chunk size, inductive vs. deductive methods, and prompting approaches. Results showed some alignment between LLM- and researcher-generated codes, with the deductive approach yielding higher and more nuanced insights (Gemma2: n = 723; Llama3.1: n = 1,042) compared to the inductive approach (Gemma2: n = 715; Llama3.1: n = 829). Approximately 45% of the LLM-generated codes provided meaningful context, though 22–39% were duplicative. While LLMs demonstrated efficiency in analyzing large volumes of textual data, nearly half of the codes lacked sufficient context or were repetitive. Our findings showed that although combining traditional qualitative analysis with AI presents a promising avenue, future research should explore how pre-trained LLMs could augment qualitative analysis with higher inter-rater reliability to improve patient-provider communication. There is a need for further research or the development of domain-specific LLMs to improve their utility in qualitative analysis.
科研通智能强力驱动
Strongly Powered by AbleSci AI