已入深夜,您辛苦了!由于当前在线用户较少,发布求助请尽量完整地填写文献信息,科研通机器人24小时在线,伴您度过漫漫科研夜!祝你早点完成任务,早点休息,好梦!

A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness

计算机科学 推论 个性化 领域(数学分析) 数据科学 领域特定语言 可信赖性 分类学(生物学) 资源(消歧) 语言模型 云计算 多归宿 领域知识 可靠性(半导体) 面子(社会学概念) 人工智能 最佳实践 域适应 知识管理 人机交互 透明度(行为) 有限的资源 授权 灵活性(工程) 推理机 延迟(音频) 移动设备
作者
Fali Wang,Zhiwei Zhang,Xianren Zhang,Zongyu Wu,Tzuhao Mo,Qiuhao Lu,Wanjing Wang,Rui Li,Junjie Xu,Xianfeng Tang,Qi He,Yao Ma,Ming S. Huang,Suhang Wang
出处
期刊:ACM Transactions on Intelligent Systems and Technology [Association for Computing Machinery]
卷期号:16 (6): 1-87 被引量:398
标识
DOI:10.1145/3768165
摘要

Large language models (LLMs) have demonstrated emergent abilities in text generation, question answering, and reasoning, facilitating various tasks and domains. Despite their proficiency in various tasks, LLMs like PaLM 540B and Llama-3.1 405B face limitations due to large parameter sizes and computational demands, often requiring cloud API use, which raises privacy concerns, limits real-time applications on edge devices, and increases fine-tuning costs. Additionally, LLMs often underperform in specialized domains such as healthcare and law due to insufficient domain-specific knowledge, necessitating specialized models. Therefore, Small Language Models (SLMs) are increasingly favored for their low inference latency, cost-effectiveness, efficient development, and easy customization and adaptability. These models are particularly well-suited for resource-limited environments and domain knowledge acquisition, addressing LLMs’ challenges and proving ideal for applications that require localized data handling for privacy, minimal inference latency for efficiency, and domain knowledge acquisition through lightweight fine-tuning. The rising demand for SLMs has spurred extensive research and development. However, a comprehensive survey investigating issues related to the definition, acquisition, application, enhancement, and reliability of SLM remains lacking, prompting us to conduct a detailed survey on these topics. The definition of SLMs varies widely; thus, to standardize, we propose defining SLMs by their capability to perform specialized tasks and suitability for resource-constrained settings, setting boundaries based on the minimal size for emergent abilities and the maximum size sustainable under resource constraints. For other aspects, we provide a taxonomy of relevant models/methods and develop general frameworks for each category to enhance and utilize SLMs effectively. We have compiled the collected SLM models and related methods on GitHub: https://github.com/FairyFali/SLMs-Survey .

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
dopamine发布了新的文献求助10
刚刚
极限001的应助被科研通管家采纳,获得100
刚刚
刚刚
Jasper的应助被科研通管家采纳,获得10
刚刚
华仔的应助被科研通管家采纳,获得10
1秒前
1秒前
共享精神的应助被科研通管家采纳,获得100
1秒前
1秒前
脑洞疼的应助被科研通管家采纳,获得10
1秒前
1秒前
隐形曼青的应助被科研通管家采纳,获得10
1秒前
SZL2026发布了新的文献求助20
1秒前
天天快乐的应助被科研通管家采纳,获得100
1秒前
1秒前
赘婿的应助被科研通管家采纳,获得10
2秒前
深情安青的应助被科研通管家采纳,获得100
2秒前
orixero的应助被科研通管家采纳,获得10
2秒前
烟花的应助被科研通管家采纳,获得10
2秒前
上官若男的应助被科研通管家采纳,获得10
2秒前
2秒前
俊逸盛男发布了新的文献求助10
2秒前
伸懒腰发布了新的文献求助10
2秒前
丘比特的应助被swimming采纳,获得10
2秒前
彭于晏的应助被科研通管家采纳,获得10
2秒前
秋风的应助被monster采纳,获得10
3秒前
lllcy发布了新的文献求助10
4秒前
Lily发布了新的文献求助10
4秒前
wanci的应助被盒子采纳,获得30
5秒前
3921337477发布了新的文献求助10
5秒前
方方完成签到,获得积分10
5秒前
一念之间的应助被史萌采纳,获得10
6秒前
6秒前
123的应助被fb12000采纳,获得10
7秒前
丘比特的应助被夏灼采纳,获得10
7秒前
3921337477发布了新的文献求助10
7秒前
8秒前
完美世界的应助被dxftx采纳,获得10
8秒前
lishuang完成签到,获得积分20
8秒前
8秒前
Akim的应助被坚定的采波采纳,获得10
9秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
八旗制度の研究 600
高分子基功能界面水蒸发材料与应用研究 500
外场增强的水汽发电研究 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8184536
求助须知:如何正确求助?哪些是违规求助? 9573526
关于积分的说明 21832132
捐赠科研通 7781791
什么是DOI,文献DOI怎么找? 3434376
关于科研通互助平台的介绍 2558490
邀请新用户注册赠送积分活动 2480989