Hierarchical Task Offloading for Vehicular Fog Computing Based on Multi-Agent Deep Reinforcement Learning

Vehicular fog computing (VFC) has been expected as a promising architecture that can make full use of computing resources of idle vehicles to increase computing capability. However, most current VFC architectures only focus on the local region and ignore the spatio-temporal heterogeneity of computing resources, resulting in that some regions have idle computing resources while others cannot satisfy the requirements of tasks. To further improve the overall computing resource utilization in the whole network, in this work, we propose a hierarchical VFC architecture, where neighboring regions can share their idle computing resources. Considering the high complexity of both inter- and intra-region cooperative task offloading in such a hierarchical VFC architecture, we put forward a distributed task offloading strategy based on multi-agent reinforcement learning in which the multi-agent reinforcement learning method is designed to learn each task vehicle’s offloading strategy in a distributed manner. Moreover, to tackle the inefficiency caused by the multi-agent credit assignment problem, we provide the counterfactual multi-agent reinforcement learning approach which exploits a counterfactual baseline to evaluate the action of each agent. Simulation results validate that the proposed hierarchical VFC architecture can effectively improve the global task computing efficiency and the proposed mechanism outperforms the baseline algorithms.