Markov Decision Process-Based Distributed Conflict Resolution for Drone Air Traffic Management

Ensuring safety and providing timely conflict alerts to small unmanned aircraft, commonly known as drones, is important to their integration into civil airspace. This paper proposes a short-term conflict avoidance algorithm for an automated low-altitude, small unmanned aircraft traffic management system. The goal is to balance between aircraft safety and efficiency subject to uncertainty in the environment, aircraft, and pilot response. The proposed algorithm generates advisories for each aircraft to follow, and is based on decomposing a large multiagent Markov decision process and fusing their solutions. As a result, the method scales well and resolves conflicts efficiently. Further, this paper presents a massively distributed architecture used to implement the conflict avoidance algorithm on a practical scale in which simultaneous conflicts can be solved in parallel in tens of milliseconds. Our controller significantly outperforms two baseline algorithms based on uncoordinated and closest-threat heuristics. In terms of conflict probability, our simulations demonstrate that our method is an order of magnitude better than the latter and about 10% better than the former.