Sequence matching for Image-Based UAV-to-Satellite Geolocalization

UAV-to-satellite geolocalization offers accurate drift-free navigation in the absence of external positioning signals. Increased deep-learning-based approaches have demonstrated their potential for high accuracy by framing the problem as a one-to-all retrieval task. However, in real-world scenario, the problem is not just a one-to-all retrieval task, which leads to a gap between research and applications. Based on this observation, We attempt to look closer to the problem instead of designing sophisticated network architectures or objective functions. In this study, we proposed a flexible and simple coarse-to-fine sequence-matching solution with targeted joint use of deep learning and classical machine learning approaches. Our goal is to improve geolocalization accuracy by matching UAV images with a few relevant reference image patches instead of all images. To this end, we first coarsely constructed a sequence of reference satellite image patches corresponding to the UAV trajectory, for which we proposed a deep feature- and manifold learning-based image-sorting method. Once the reference satellite patches are sorted and aligned with the UAV trajectory, the reference sequence is determined. Given a query UAV frame, the search area can be decreased from two to one dimensions. In particular, both deep-learning-based and classical image-matching algorithms can provide competitive accuracy when integrating sequence constraints. We demonstrate that classical manifold learning-based and image matching methods perform exceptionally well for UAV-to-satellite geolocalization when utilized jointly with suitable deep learning techniques. We validated the approach’s unique outperformance on two challenging and realistic UAV-to-satellite geolocalization datasets. Dataset, code and models are available for research purposes at https://seqmatch.geovisuallocalization.com/.