计算机科学
推论
人工智能
钥匙(锁)
系统工程
过程(计算)
工程类
鉴定(生物学)
背景(考古学)
工作(物理)
作者
Jingwei Cai,Dehao Kong,Hantao Huang,Zishan Jiang,Zixuan Ma,Qingyu Guo,Zhenxing Zhang,Guiming Shi,Mingyu Gao,Kaisheng Ma,Minghui Yu
出处
期刊:Proceedings
[Institute of Electrical and Electronics Engineers]
日期:2026-01-31
卷期号:: 1-19
被引量:2
标识
DOI:10.1109/hpca68181.2026.11408526
摘要
As a major provider of LLM inference services, ByteDance has continuously explored diverse accelerator options to meet the rapidly growing inference demands of various heterogeneous LLM scenarios with higher cost-effectiveness, thereby enabling LLMs to serve more people worldwide. However, during this process, we have found that the complexity and opacity of cloud scenarios and corresponding cloud accelerators make it difficult for academia and many innovative chip startups to fully understand the real demands and challenges of these scenarios, which in turn severely restricts innovation and application potential in this field. To bridge this gap, we first present and analyze the data and characteristics of the ByteDance Doubao LLM app across multiple dimensions, helping the community understand real-world cloud scenarios, and detail the challenges and opportunities we have identified. Second, we propose and plan to open-source our multi-level evaluation framework, XPU-Perf, which includes benchmarks spanning instructions, operators, and models. This framework improves interpretability and trustworthiness, and helps promising new accelerator architectures gain wider adoption and development. Finally, we present comparative results of four typical accelerators, summarize their shortcomings and challenges, conduct in-depth analysis, and highlight numerous architectural and scheduling innovation opportunities we have observed.
科研通智能强力驱动
Strongly Powered by AbleSci AI