Optimal Attack Path Planning based on Reinforcement Learning and Cyber Threat Knowledge Graph Combining the ATT&CK for Air Traffic Management System

With the development of the Air Traffic Management System (ATM), the ATM network has transitioned from a closed physical isolation network to an open Cyber-Physical System (CPS). As a result, the surface of cyber-attacks is constantly expanding, and the degree of automation and correlation of attacks is constantly increasing. Traditional penetration testing attack path planning relies on the expertise of specific field experts, which can be time-consuming and resource-intensive when dealing with large networks. A large number of useful cyber security information are seriously fragmented and not integrated. In this paper, we aim to address these challenges by constructing a novel ATM system cyber threat knowledge graph called ATMCyKG. This knowledge graph is based on ATT&CK attack TTP style templates, including attack tactics, techniques, and processes. It defines entities and attributes, and their relationships. By combining ATT&CK tactics and techniques with the knowledge graph, we propose an attack path planning method based on ATMCyKG and integrate it into a reinforcement learning model. We use a variety of reinforcement learning algorithms to conduct comparative experiments and conduct a panoramic analysis of its attack process. Finally, the experimental results of the three reinforcement algorithms are analyzed and summarized. This paper introduces the ATMCyKG for the first time and utilizes Neo4j for its construction. From the perspective of an attacker, we utilized reinforcement learning to discover vulnerability sequences by selecting effective action sequences to achieve the desired target. By planning automatic attack paths, we can eliminate the reliance on expert experience, save manpower and time, and improve the operability and testing efficiency of automated penetration testing in ATM. This is of significant importance in ensuring aviation transportation safety and maintaining airspace order.