SMART-TRACK: A Novel Kalman Filter-Guided Sensor Fusion for Robust UAV Object Tracking in Dynamic Environments

In the field of sensor fusion and state estimation for object detection and localization, ensuring accurate tracking in dynamic environments poses significant challenges. Traditional methods, such as the Kalman filter (KF), often fail when measurements are intermittent, leading to rapid divergence in state estimations. To address this, we introduce sensor measurement augmentation and reacquisition tracker (SMART), a novel approach that leverages high-frequency state estimates from the KF to guide the search for new measurements, maintaining tracking continuity even when direct measurements falter. This is crucial for dynamic environments where traditional methods struggle. Our contributions include the following. First, versatile measurement augmentation using KF feedback: we implement a versatile measurement augmentation system that serves as a backup when primary object detectors fail intermittently. This system is adaptable to various sensors, demonstrated using depth cameras where KF’s 3-D predictions are projected into 2-D depth image coordinates, integrating nonlinear covariance propagation techniques simplified to first-order approximations. Second, open-source ROS2 implementation: we provide an open-source ROS2 implementation of the SMART-TRACK framework, validated in a realistic simulation environment using Gazebo and ROS2, fostering broader adaptation and further research. Our results showcase significant enhancements in tracking stability, with estimation root-mean-squared error (RMSE) as low as 0.04 m during measurement disruptions, advancing the robustness of UAV tracking and expanding the potential for reliable autonomous UAV operations in complex scenarios. The implementation is available at https://github.com/mzahana/d2dtracker_drone_detector.