计算机科学
推论
云计算
资源配置
GSM演进的增强数据速率
分布式计算
人工智能
计算机网络
操作系统
作者
Ying He,Jingcheng Fang,Fei Richard Yu,Victor C. M. Leung
标识
DOI:10.1109/tmc.2024.3415661
摘要
With the increasing popularity and demands for large language model applications on mobile devices, it is difficult for resource-limited mobile terminals to run large-model inference tasks efficiently. Traditional deep reinforcement learning (DRL) based approaches have been used to offload large language models (LLMs) inference tasks to servers. However, existing DRL solutions suffer from data inefficiency, insensitivity to latency requirements, and non-adaptability to task load variations, which will degrade the performance of LLMs. In this paper, we propose a novel approach based on active inference for LLMs inference task offloading and resource allocation in cloud-edge computing. Extensive simulation results show that our proposed method has superior performance over mainstream DRLs, improves in data utilization efficiency, and is more adaptable to changing task load scenarios.
科研通智能强力驱动
Strongly Powered by AbleSci AI