Pseudo-LiDAR for Visual Odometry
Yanzi Miao, Huiying Deng, Chaokang Jiang, Zhiheng Feng, Xinrui Wu, Guangming Wang, Hesheng Wang
Abstract
As one of the important tasks in the field of robotics and machine vision, visual odometry provides tremendous help for various applications such as navigation, location, etc. Conventionally, the task of visual odometry mainly relies on the input of continuous images. However, it is very complicated for the odometry network to learn the epipolar geometry information provided by the images. Since the 6 degree of freedom (DoF) poses transformation occurs in 3D space and learning poses from 3D point clouds is more straightforward, this paper introduces the concept of pseudo-LiDAR to the odometry task. The pseudo-LiDAR point cloud is formed by back-projecting the depth map generated from the image into 3D space. Due to the limitation of calculation power, most current algorithms based on the point cloud need to sample 8192 points from the point cloud as input, but such an approach makes the rich point cloud information in the pseudo-LiDAR point cloud not fully utilized. To address this problem, a projection-aware algorithm is adopted, which achieves efficient point cloud learning and improves the accuracy of the network while preserving the 3D structure information in the pseudo-LiDAR point cloud. Finally, an image-only 2D-3D fusion module is proposed to enhance the pseudo-LiDAR point features using information such as the texture and color of the images. Through multimodal fusion, the network achieves a deeper understanding of the environment. Experiments on the KITTI dataset prove the effectiveness of our method. The source code will be open-sourced at https://github.com/IRMVLab/Pseudo-LiDAR-for-Visual-Odometry.