V-VformerVIO: vital vision transformer–guided visual–inertial odometry

In the field of multisensor fusion technology, visual–inertial odometry (VIO) aims to provide accurate pose information through self-motion estimation, which is crucial for high-precision and real-time localization. However, all existing approaches are unable to satisfy deployability and real-time requirements while providing prolonged positioning services. Therefore, we propose a VIO approach, named V-VformerVIO, which ensures localization accuracy while significantly reducing the length of the input sequence. First, a new two-stage transformer module is designed to learn accurate visual features. The first-stage transformer focuses on the temporal consistency of features among sequence images, and the second-stage transformer uses a mask mechanism to associate the spatiotemporal correspondence of features, ultimately achieving context-aware feature enhancement. Furthermore, owing to the inferior feature extraction performance of visual modules in complex environments, a Conv1D-based inertial feature encoder is designed to extract features for enhancing feature stability. Meanwhile, to prevent inconsistencies in the fusion of inertial and visual features, a feature fusion module is applied to fuse extracted features. Ultimately, localization information is acquired via a position regression module. Through a large number of experimental results, our approach can shorten the length of feature acquisition to S=3, and the consumption of on-chip resources is decreased by 20.07% in contrast to state-of-the-art algorithms, which exhibits strong competitiveness.