计算机科学
软件部署
推论
延迟(音频)
工作量
边缘设备
量化(信号处理)
模式(遗传算法)
GSM演进的增强数据速率
边缘计算
实时计算
计算机工程
嵌入式系统
解码方法
分布式计算
高效能源利用
计算机硬件
粒度
并行计算
综合数据
搜索引擎索引
功率消耗
偏移量(计算机科学)
利用
模糊推理系统
电效率
钥匙(锁)
计算机体系结构
标准化
试验数据
作者
Amir Taherin,Sana Taghipour Anvari,Charles Amante,Yixiao Chen,Ruben Noroian,Zlatan Feric,Nícolas Bohm Agostini,Pu Zhao,José Cano,Bin Ren,Yanzhi Wang,David R Kaeli
出处
期刊:IEEE International Symposium on Workload Characterization
[Institute of Electrical and Electronics Engineers]
日期:2026-09-01
卷期号:: 471-485
被引量:2
标识
DOI:10.1109/iiswc71020.2026.00048
摘要
Edge LLM deployment is shaped by more than model size and precision: inference backend, hardware platform, memory traffic, and power management all affect latency and efficiency. We present Hydra, a common-schema, phase-aware workload characterization framework for LLM inference on edge SoCs. Hydra instruments HuggingFace Transformers and llama.cpp with a shared per-prompt timing schema and fuses those records with hardware telemetry, enabling a multi-dimensional characterization of performance, system-resource utilization, and efficiency across prefill and decode phases. Using Hydra, we evaluate three consecutive edge System-on-Chip (SoC) generations (AGX Xavier, AGX Orin, and AGX Thor), 13 instruction-tuned LLMs from seven families, five execution formats, and consider input/output-length sensitivity. The resulting artifact contains roughly 107K per-prompt records and is publicly released with Hydra. Our analysis shows that aggregate latency alone hides key deployment effects: backend structure changes where latency is introduced, quantization reduces memory traffic and energy but does not predict power monotonically, and SoC generation changes how utilization and efficiency should be interpreted. By connecting phase-level timing with system-resource utilization and efficiency metrics, Hydra enables reproducible, phase-aware characterization of edge LLM inference. Hydra's source code and the collected per-prompt trace corpus are available open-source at: https://github.com/amirtaherin/hydra
科研通智能强力驱动
Strongly Powered by AbleSci AI