Reinforcement Learning-Based Optimization Framework for Application Component Migration in NFV Cloud-Fog Environments

By decoupling network functions from the underlying hardware, Network Function Virtualization (NFV) allows application components to be implemented as sets of Virtual Network Functions (VNFs) chained in a specific order, represented by VNF-Forwarding Graphs (VNF-FG). Fog computing is instrumental to tap into the full potential of NFV by deploying VNFs in close proximity to end-users, thus decreasing the latency significantly. However, the mobility of end-users and the fog nodes, and the limited fog nodes coverage results in service discontinuity and may increase application delay. Application component migration offers great potential to address this issue. In this paper, we propose a component migration strategy in an NFV-based hybrid cloud/fog system considering the mobility of both end-users and fog nodes. We use the Gauss-Markov mobility model and a random walk mobility model for fog nodes and end-user devices, respectively. We modeled the problem mathematically, which minimizes the aggregated weighted function of application delay and cost. However, considering the mobility of both end-users and fog nodes makes the problem quite complex. Hence, we propose a Deep Reinforcement Learning (DRL) approach to decide where and when to migrate application components and to achieve rapid decision-making. Simulation results demonstrate that the proposed scheme performs well. It offers favorable convergence and outperforms existing algorithms in terms of application delay and migration costs.