Uncertainty-Based Adaptive Sensor Fusion for Visual-Inertial Odometry under Various Motion Characteristics

We propose an uncertainty-based sensor fusion framework for visual-inertial odometry, which is the task of estimating relative motion using images and measurements from inertial measurement units. Visual-inertial odometry enables robust and scale-aware estimation of motion by incorporating sensor states, such as metric scale, velocity, and the direction of gravity, into the estimation. However, the observability of the states depends on sensor motion. For example, if the sensor moves in a constant velocity, scale and velocity cannot be observed from inertial measurements. Under these degenerate motions, existing methods may produce inaccurate results because they incorporate erroneous states estimated from non-informative inertial measurements. Our proposed framework is able to avoid this situation by adaptively switching estimation modes, which represents the states that should be incorporated, based on their uncertainties. These uncertainties can be obtained at a small computational cost by reusing the Jacobian matrices computed in bundle adjustment. Our approach consistently outperformed conventional sensor fusion in datasets with different motion characteristics, namely, the KITTI odometry dataset recorded by a ground vehicle and the EuRoC MAV dataset captured from a micro aerial vehicle.