An Attention-Aware Video and ADS-B Fusion Method for Airport Surface Aircraft Detection and Tracking

Airport surface surveillance is a critical component for ensuring aviation safety and enhancing operational efficiency, with the identification of surface aircraft at the core of airport surface surveillance, encompassing the detection and tracking of aircraft. Traditional video-based methods heavily rely on object features, making it prone to missing small objects at a distance. Other surveillance sources have the potential to compensate for the deficiencies inherent in video-based systems. The integration of video data with these alternative surveillance sources, through a synergistic fusion approach, holds promise in resolving the aforementioned challenges. However, existing multi-sensor fusion approaches cannot be directly applied, considering the heterogeneous spatiotemporal characteristics and unreliability in ADS-B and video. To this end, we propose an Attention-aware Video and ADS-B Fusion (AaVAF) method for airport surface aircraft detection and tracking. Firstly, a coarse-to-fine cross-modal matching strategy is designed to temporally and spatially align heterogeneous video and ADS-B data, providing a data foundation for subsequent fusion. Secondly, to extract key information from a vast amount of unreliability data and achieve effective fusion, an attention-aware multi-scale fusion module is proposed. Finally, the identification and tracking of aircraft objects on the airport surface are achieved through a two-stage association method utilizing the fused features. Experiments on real-world airport surface surveillance videos and ADS-B data demonstrate that the proposed fusion method outperforms baselines in detecting and tracking airport surface objects, especially in recognizing small objects at long distances in panoramic scenes.