计算机科学
微服务
程序设计语言
软件工程
软件
操作系统
集合(抽象数据类型)
故障排除
钥匙(锁)
嵌入式系统
作者
Jovan Stojkovic,Abraham Farrell,Zhangxiaowen Gong,Christopher J. Hughes,Josep Torrellas
出处
期刊:Proceedings
[Institute of Electrical and Electronics Engineers]
日期:2026-01-31
卷期号:: 1-17
被引量:3
标识
DOI:10.1109/hpca68181.2026.11408580
摘要
Microservices suffer from the execution of auxiliary operations known as datacenter tax, such as RPC and TCP processing, and data (de)serialization, (de)encryption, and (de)compression. To minimize this tax, multiple hardware accelerators have been proposed. However, it is unclear how these accelerators should be orchestrated. Past work has focused only on orchestrating accelerators in coarse-grained environments with monolithic applications. In this paper, we characterize the needs of orchestrating an ensemble of on-package accelerators in microservice environments. We observe that orchestration frameworks need to be highly dynamic and nimble. The basic operations to be accelerated are fine grained, potentially taking only tens of$\mu \mathrm{s}$. Moreover, the sequence of accelerators to use is often affected by “branch conditions” whose real-time resolution determines the set of subsequent accelerators needed. To address these challenges, we present AccelFlow, the first orchestration framework for onpackage accelerators of microservices. In AccelFlow, CPU cores build software structures called Traces that contain sequences of accelerators to call. A core enqueues a trace in an accelerator in user mode and, from then on, the accelerators in the trace execute in sequence without CPU involvement. A trace can include branch conditions whose outcomes determine the trace control flow. Compared to state-of-the-art accelerator orchestrators, AccelFlow on average reduces P99 tail latency by 70 %, reduces average latency by 38 %, and increases throughput by 120 %.
科研通智能强力驱动
Strongly Powered by AbleSci AI