Multiagent Federated Deep-Reinforcement-Learning-Based Collaborative Caching Strategy for Vehicular Edge Networks

With the rapid advancement of in-vehicle communication technology, vehicular edge caching has garnered considerable attention as a pivotal technology to improve the efficiency of data transmission. However, existing studies often overlook the issues of increased average content access latency and decreased caching hit rate, stemming from the conflict between limited storage space in in-vehicle edge servers and vehicle mobility. To address these issues, this paper proposes a Multi-agent Federated Deep Reinforcement Learning based Collaborative Caching Strategy (MFDRL-CCS), leveraging Vehicle-to-Vehicle (V2V) communications. Specifically, we first perform vehicle connectivity prediction based on Recurrent Neural Network (RNN) considering the characteristics of vehicle nodes and their interrelations. Then, the optimal caching vehicle is selected based on the connectivity between vehicle nodes and the density of vehicle nodes. Meanwhile, a Multi-Head Attention Popularity Prediction (MHAPP) model is also constructed, which amalgamates multi-dimensional features, including historical popularity, social relationships, and geographic location, to predict content popularity. Finally, the edge collaborative caching model is formulated as a Markov Decision Process (MDP). Under the multi-agent competitive deep Q-learning framework, each vehicle learns the optimal caching strategy through an independent Q-network to maximize long-term rewards, and uses federated learning to train the caching replacement algorithm in a distributed manner. Compared to existing caching policies, the caching policy proposed in this paper improves the caching hit rate by approximately 19.8% and reduces the content access latency by about 12.5%.