Multi-Agent Deep Reinforcement Learning for Resource Scheduling in Hybrid-Energy MEC Systems Under Long-Term Constraints

In a mobile edge computing environment where mobile devices are powered by green energy, minimizing power consumption and latency under long-term MEC energy constraints faces several challenges: unpredictable tasks, keeping both the privacy of the mobile device’s battery and efficient task scheduling, stabilization of power consumption and latency accumulation, and non-convex optimization for a continuous-discrete decision space. To tackle these challenges, we investigate the latency-aware resource-constrained scheduling (LARCS) problem in a hybrid-energy edge-cloud MEC system to minimize energy consumption and latency under long-term MEC energy constraints. For such a scenario, for the first time, we address protecting battery privacy and mitigating energy rebound peaks, which indicate sudden power surges from concurrent high-load tasks. We model the LARCS using mixed integer nonlinear programming (MINLP) and prove its NP-completeness. Then, it is reformulated as a Markov game. We adopt the multi-actor-attention-critic learning mechanism to preserve user privacy and employ a reward shape in reward functions for mitigating energy rebound peaks. On the above basis, we propose a latency-aware resource-constrained scheduling algorithm on the basis of multi-agent deep reinforcement learning (LARC-MADRL) to coordinate optimizations for multiple mobile users. This proposed algorithm requires no prior knowledge of uncertain parameters, operates independently of hybrid-energy dynamic models, and maintains user privacy. Numerical simulations validate that our proposed algorithm outperforms benchmark algorithms in minimizing power consumption, latency, and dropout rate while achieving long-term benefits, better convergence, improved stability, and enhanced scalability.