Joint Service Migration and Resource Allocation in Edge IoT System Based on Deep Reinforcement Learning

Multiaccess edge computing (MEC) provides services for resource-sensitive and delay-sensitive Internet of Things (IoT) applications by extending the capabilities of cloud computing to the edge of the networks. However, the high mobility of IoT devices (e.g., vehicles) and the limited resources of edge servers (ESs) affect the service continuity and access latency. Service migration and reasonable resource (re-)allocation consequently become needed to ensure Quality of Service (QoS). However, service migration results in additional latency. In addition, different mobile IoT users have different resource requirements and different resource allocation policies of target ESs also determine whether service migration is necessary. Subsequently, how to jointly optimize service migration and resource allocation (SMRA) is a challenge that needs to be carefully addressed. To this end, this article investigates the joint optimization problem of SMRA in MEC environments to minimize the access delay of IoT users. It proposes a joint SMRA algorithm based on deep reinforcement learning (DRL), which takes into account the mobility of IoT users and decides whether to migrate services, where to migrate, and how to allocate resources through the long short time memory (LSTM) algorithm and the parameterized deep$Q$-network (PDQN) algorithm. Moreover, the PDQN algorithm effectively solves the discrete-continuous hybrid action space challenge in the SMRA problem. Finally, we conduct evaluation using a real-world data set of Beijing cab trajectories to verify the effectiveness and superiority of our proposed SMRA solution.