Knowledge-Aided Model-Based Reinforcement Learning for Anti-Jamming Strategy Learning

With the development of electronic warfare, the jammer is becoming much smarter than before and its capability of learning and adapting to the radar's transmission strategy poses a great challenge to the radar. The learning-based methods have been recently proposed to improve the anti-jamming performance of the ground-based radar when dealing with smart mainlobe self-protection jammers. Model-free reinforcement learning (MFRL) is a widely used learning-based method that is easy to implement and can achieve a good performance. However, MFRL methods have low sample efficiency and a substantial number of training samples are needed to achieve an acceptable performance. In this paper, a knowledge aided model-based RL (MBRL) approach is developed to improve the sample-efficiency of the anti-jamming strategy learning methods. To avoid learning from scratch and improve learning efficiency, commonly used jamming strategies are treated as prior knowledge and are parameterized by neural networks with different parameters. Based on that, the underlying unknown jamming strategy is approximately expressed by the weighted combination of these prior models and MBRL is used to simultaneously learn the anti-jamming strategy and the unknown jamming strategy. This learning task is formulated as a bilevel nested optimization problem and an online-offline mixed approach is proposed to solve the problem. Simulation results show that the radar can learn an acceptable performance with only 2,000 online interaction samples compared with the MFRL methods regardless of whether the unknown jamming strategy is involved in the prior knowledge.