Techniques, Performance, and Feasibility of Natural Language Processing for Abstract Screening in Evidence Synthesis: A Systematic Review

计算机科学 人工智能 系统回顾 机器学习 自然语言处理 引用 预印本 工作量 灰色文学 情报检索 过程(计算) 钥匙(锁) 数据科学 选择(遗传算法) 语言模型 人工智能应用 降维 英语 深度学习
作者
Ravi Shankar,Ziyu Goh,Qian Xu
出处
期刊:Campbell Systematic Reviews [The Campbell Collaboration]
卷期号:22 (3): 18911803261469813-18911803261469813
标识
DOI:10.1177/18911803261469813
摘要

Background: Natural language processing (NLP) techniques offer promising solutions for semi-automating the time-consuming process of abstract screening in systematic reviews. The exponential growth of published literature has created significant bottlenecks, with review teams manually assessing thousands of abstracts over weeks to months. Single reviewers can miss 5-13% of relevant studies, necessitating dual screening that further increases workload. Advances in artificial intelligence, including deep learning models such as BERT and its successors, show potential for automating this critical step, but comprehensive evidence on optimal approaches, performance, and practical feasibility remains limited. Objectives: This systematic review aimed to assess techniques, performance, and feasibility of NLP approaches for title and abstract screening by characterizing the range of NLP methods used, summarizing performance on key metrics like workload reduction and recall, evaluating real-world implementation feasibility, and identifying research gaps and future directions. Search Methods: We searched PubMed, Web of Science, Embase, CINAHL, The Cochrane Library, Scopus, and gray literature sources from inception to December 2024. The search strategy, developed with an information specialist and peer-reviewed using PRESS guidelines, targeted keywords related to natural language processing, machine learning, abstract screening, and systematic reviews. Additional sources included conference proceedings, preprint servers, reference lists, forward citation tracking, and expert consultation. Selection Criteria: We included primary studies of any design describing development or evaluation of NLP techniques for automating title and abstract screening in evidence syntheses. Eligible studies reported on NLP methods, screening performance (workload reduction, recall, precision), or implementation feasibility. Studies using only rule-based approaches without machine learning, systematic reviews of NLP methods, commentaries, and conference abstracts were excluded. No language or date restrictions were applied. Data Collection and Analysis: Two reviewers independently screened titles, abstracts, and full texts using Covidence software, with disagreements resolved through discussion. Data extraction covered study characteristics, NLP techniques, training approaches, performance metrics, and feasibility considerations. Risk of bias was assessed using a modified ROBIS tool. Given diverse techniques and outcomes, we conducted narrative synthesis following SWiM guidelines, grouping studies by NLP approach. Main Results: From 4,105 records, 19 studies met inclusion criteria, with 68.4% published since 2023, reflecting rapid field advancement. Studies employed diverse approaches from traditional machine learning (Support Vector Machines, Random Forests) to advanced deep learning models, particularly BERT variants. Most achieved >90% recall with workload reductions of 13-96%, representing substantial time savings. Deep learning models with transfer learning consistently outperformed traditional approaches. However, implementation faced significant barriers including requirements for high-quality training data, specialized computational resources, technical expertise, and user-friendly interfaces. Performance was generally better for targeted reviews with lower inclusion prevalence. Authors’ Conclusions: NLP techniques, especially deep learning with transfer learning, show substantial promise for semi-automating abstract screening with potential for large workload savings while maintaining high recall. However, challenges remain regarding training data quality, computational requirements, technical expertise needs, and user-centered design. Realizing full potential requires interdisciplinary collaboration to develop reliable, generalizable tools integrating seamlessly with human expertise and existing workflows. Future priorities include creating standardized datasets, conducting prospective evaluations, developing user-friendly interfaces, and establishing implementation best practices to revolutionize evidence synthesis efficiency.

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
xbnie完成签到,获得积分10
1秒前
阳阳秋完成签到,获得积分10
3秒前
4秒前
GUAN发布了新的文献求助10
4秒前
Lucas的应助被你真是那个啊采纳,获得10
4秒前
5秒前
努力加油干的小猫咪完成签到 ,获得积分10
5秒前
福yyy发布了新的文献求助10
8秒前
jianghuren发布了新的文献求助10
9秒前
Zz发布了新的文献求助10
10秒前
zxzx发布了新的文献求助10
12秒前
horse完成签到,获得积分10
12秒前
TOT完成签到,获得积分10
13秒前
天天快乐的应助被绪绪采纳,获得10
13秒前
14秒前
16秒前
16秒前
lila完成签到,获得积分10
19秒前
19秒前
19秒前
19秒前
浮生发布了新的文献求助20
20秒前
trx发布了新的文献求助10
20秒前
csy完成签到,获得积分10
21秒前
科目三的应助被孟庆亮采纳,获得10
21秒前
21秒前
21秒前
子夜歌发布了新的文献求助20
23秒前
hoyden完成签到,获得积分10
24秒前
up0818完成签到 ,获得积分10
24秒前
LQY完成签到 ,获得积分10
25秒前
shao发布了新的文献求助10
25秒前
邵蹦蹦发布了新的文献求助10
25秒前
Attendre发布了新的文献求助10
25秒前
26秒前
27秒前
27秒前
兜里有糖完成签到,获得积分10
28秒前
28秒前
28秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
八旗制度の研究 600
高分子基功能界面水蒸发材料与应用研究 500
外场增强的水汽发电研究 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8186750
求助须知:如何正确求助?哪些是违规求助? 9575027
关于积分的说明 21839240
捐赠科研通 7784331
什么是DOI,文献DOI怎么找? 3434808
关于科研通互助平台的介绍 2559206
邀请新用户注册赠送积分活动 2481611