计算机科学
任务(项目管理)
弹丸
边距(机器学习)
钥匙(锁)
零(语言学)
模板
语言模型
简单(哲学)
自然语言
人工智能
自然语言处理
程序设计语言
机器学习
语言学
哲学
管理
经济
认识论
计算机安全
有机化学
化学
作者
Jason Wei,Maarten Bosma,Vincent Y. Zhao,Kelvin Guu,Adams Wei Yu,Lester, Brian,Nan Du,Andrew M. Dai,Quoc Viet Le
标识
DOI:10.48550/arxiv.2109.01652
摘要
This paper explores a simple method for improving the zero-shot learning abilities of language models. We show that instruction tuning -- finetuning language models on a collection of tasks described via instructions -- substantially improves zero-shot performance on unseen tasks. We take a 137B parameter pretrained language model and instruction-tune it on over 60 NLP tasks verbalized via natural language instruction templates. We evaluate this instruction-tuned model, which we call FLAN, on unseen task types. FLAN substantially improves the performance of its unmodified counterpart and surpasses zero-shot 175B GPT-3 on 20 of 25 tasks that we evaluate. FLAN even outperforms few-shot GPT-3 by a large margin on ANLI, RTE, BoolQ, AI2-ARC, OpenbookQA, and StoryCloze. Ablation studies reveal that number of finetuning datasets, model scale, and natural language instructions are key to the success of instruction tuning.
科研通智能强力驱动
Strongly Powered by AbleSci AI