VIMOT: A Tightly Coupled Estimator for Stereo Visual-Inertial Navigation and Multiobject Tracking
Shaoquan Feng, Xingxing Li, Chunxi Xia, Jianchi Liao, Yuxuan Zhou, Shengyu Li, Xianghong Hua
Abstract
Most existing visual-inertial navigation system (VINS) simply rejects dynamic object features to improve navigation performance, rendering the loss of dynamic object motion information. Whereas some emerging applications, such as autonomous driving, need to accurately perceive the movement of surrounding objects for making decisions. Aiming at performing accurate pose estimations for both the ego vehicle and surrounding objects, we propose a tightly-coupled estimator for visual-inertial navigation and multi-object tracking, called VIMOT. The proposed system directly utilizes a 3D bounding box generated by an object detector to represent an object. On this basis, a multi-level data association method together with a geometry-based dynamic object classification method is applied for object tracking. Instead of rejecting the features of dynamic objects, their visual features are fully exploited in our system to form the object-related factors, which are jointly optimized with the static visual feature factors and the IMU pre-integration factors for both ego-vehicle and multi-object state estimation. The performance of the proposed system is evaluated on both public datasets and real-world experiments. Compared with existing state-of-the-art similar methods, the results show that integrating multi-object tracking into VINS can significantly improve the pose accuracy of both the VINS and tracked objects, which makes both estimations benefit from each other.