计算机科学
变压器
语言模型
任务(项目管理)
微调
人工智能
机器学习
自然语言处理
多任务学习
源代码
计算
物理
管理
算法
量子力学
电压
经济
操作系统
作者
Niall Taylor,Yi Zhang,Dan W. Joyce,Zhiqiang Gao,Andrey Kormilitzin,Alejo Nevado-Holgado
出处
期刊:IEEE transactions on neural networks and learning systems
[Institute of Electrical and Electronics Engineers]
日期:2024-01-01
卷期号:: 1-11
被引量:9
标识
DOI:10.1109/tnnls.2023.3294633
摘要
When the first transformer-based language models were published in the late 2010s, pretraining with general text and then fine-tuning the model on a task-specific dataset often achieved the state-of-the-art performance. However, more recent work suggests that for some tasks, directly prompting the pretrained model matches or surpasses fine-tuning in performance with few or no model parameter updates required. The use of prompts with language models for natural language processing (NLP) tasks is known as prompt learning. We investigated the viability of prompt learning on clinically meaningful decision tasks and directly compared this with more traditional fine-tuning methods. Results show that prompt learning methods were able to match or surpass the performance of traditional fine-tuning with up to 1000 times fewer trainable parameters, less training time, less training data, and lower computation resource requirements. We argue that these characteristics make prompt learning a very desirable alternative to traditional fine-tuning for clinical tasks, where the computational resources of public health providers are limited, and where data can often not be made available or not be used for fine-tuning due to patient privacy concerns. The complementary code to reproduce the experiments presented in this work can be found at https://github.com/NtaylorOX/Public_Clinical_Prompt.
科研通智能强力驱动
Strongly Powered by AbleSci AI