Deep Reinforcement Learning-Based Periodic Earth Observation Scheduling for Agile Satellite Constellation

The development of remote sensing satellite constellations has created an increasing need for periodic observations of ground targets. At present, the research on periodic observation is not sufficient, and the multisatellite scheduling algorithm based on deep reinforcement learning (DRL) is still relatively few and is trained inefficiently. In this paper, a periodic observation mission planning (POMP) algorithm is proposed for the periodic Earth observation scheduling problem. The POMP is based on the encoder/decoder architecture. First, static and dynamic attributes of the observation task and satellite are encoded using three convolutional networks in the encoder and decoder. Second, an additive attention mechanism is employed to calculate the probability of each observation task being selected. Third, the model is trained with the REINFORCE with rollout baseline algorithm. Experimental results in various scenarios show that both training with the REINFORCE with rollout baseline algorithm and encoding dynamic attributes of the satellite as part of the step context vector are effective. The proposed POMP can maintain the gap with the ant colony optimization (ACO) algorithm in weight degree of timeout (WDT) and total timeout within 0.7% and 1.5 h, respectively, while having a tens of seconds advantage over ACO in terms of computational time.