Towards End-to-end Learning of Visual Inertial Odometry with an EKF
Chunshang Li, Steven L. Waslander
Abstract
Classical visual-inertial fusion relies heavily on manually crafted image processing pipelines, which are prone to failure in situations with rapid motion and texture-less scenes. While end-to-end learning methods show promising results in addressing these limitations, embedding domain knowledge in the form of classical estimation processes within the end-to-end learning architecture has the potential of combining the best of both worlds. In this paper, we propose the first end-to-end trainable visual-inertial odometry (VIO) algorithm that leverages a robo-centric Extended Kalman Filter (EKF). The EKF propagates states through a known inertial measurement unit (IMU) kinematics model and accepts relative pose measurements and uncertainties from a deep network as updates. The system is fully differentiable and can be trained end-to-end through backpropagation. Our method achieves a translation error of 1.27% on the KITTI odometry dataset, which is competitive among classical and learning VIO methods. The implementation is publicly available on GitHub <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>