计算机科学
调度(生产过程)
算法
人工智能
公平份额计划
数据挖掘
理论计算机科学
作业车间调度
实时计算
单调速率调度
数学优化
算法设计
作者
Eunyeong Cho,Jehyeon Bang,Ranggi Hwang,Minsoo Rhu
出处
期刊:Proceedings
[Institute of Electrical and Electronics Engineers]
日期:2026-01-31
卷期号:: 1-16
被引量:4
标识
DOI:10.1109/hpca68181.2026.11408493
摘要
The emergence of reasoning-based LLMs leveraging Chain-of-Thought (CoT) inference introduces new serving challenges, as their extended reasoning phases delay user-visible output and inflate Time-To-First-Token (TTFT). Existing LLM serving frameworks fail to distinguish between reasoning and answering phases, leading to performance degradation under GPU memory constraints. We present PASCAL, a phase-aware scheduling algorithm that prioritizes reasoning to reduce TTFT while using controlled preemption and token pacing during answering to preserve Quality-of-Experience (QoE). Our hierarchical scheduler combines instance-level placement with intrainstance execution and enables dynamic migration at phase boundaries to balance load and reduce interference. Across benchmarks using DeepSeek-R1-Distill-Qwen-32B, Pascal reduces tail TTFT by up to 72 % while maintaining answering phase SLO attainment, demonstrating the importance of phaseaware scheduling for reasoning-based LLM deployment.
科研通智能强力驱动
Strongly Powered by AbleSci AI