Deep-Reinforcement-Learning-Based Resource Allocation for MEC-Assisted Satellite–Terrestrial Integrated Networks

This paper investigates the mixed-timescale resource allocation problem in satellite-terrestrial integrated networks (STIN). Moreover, the multi-access edge computing (MEC) technology and millimeter wave (mmWave) with rich spectrum resource are merged into the STIN to improve the network performance. A network utility maximization problem characterized by the achievable rate and backhaul reduction is formulated under the constraints of the maximum caching capacity, transmission power of mmWave small-cell base stations (SBSs) and quality of service (QoS) for Internet of Things (IoT) devices, where the caching placement, power allocation and user-SBS association are jointly optimized. In order to tackle this mixed-integer nonlinear programming (MINLP) problem, we decompose the original problem into the long-term caching placement subproblem, and short-term power allocation and user-SBS association subproblems. Then, a multi-agent deep reinforcement learning (MADRL)-based independent proximal policy optimization (IPPO) algorithm is proposed to solve the short-term user-SBS association subproblem. Meanwhile, the linear programming (LP) is used to solve the long-term caching placement subproblem. Furthermore, we derive the closed-form solution of the short-term power allocation subproblem through the Karush-Kuhn-Tucker (KKT) conditions. Simulation results are carried out to validate the effectiveness and scalability of the proposed joint approach.