Deep reinforcement learning method for satellite range scheduling problem

The satellite range scheduling problem (SRSP) is a range of combinatory optimization, which plays a vital role in the regular operation and mission accomplishment of in-orbit satellites. However, with the increase in the number of satellites and the client requirements, there is some limitation in dealing with the SRSP for existing methods, especially on large-scale problems. Therefore, we propose a deep reinforcement learning (DRL) method, which is integrated into a heuristic scheduling method for the satellite range scheduling problem. The core idea of the algorithm is to decompose the problem into two subproblems: (1) Assignment problem, which assigns each task on different antennas. (2) Single antenna scheduling problem, which determines the execution start and end time of selected tasks on the antenna. The two subproblems are performed iteratively and modeled as a general paradigm. In the paradigm, the DRL is to determine the process of task assignment, and the heuristic scheduling method can quickly solve the single antenna scheduling problem. The objective function of the scheduling problem is to maximize the total reward. The DRL updates the gradient information based on the reward obtained by the heuristic scheduling method. To verify this idea, various scale experiments are considered to examine the performance of training scenarios. Experimental results show that the proposed paradigm combining DRL with a heuristic scheduling method can effectively deal with the SRSP.