An Effective Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds
Chaoda Zheng, Xu Yan, Haiming Zhang, Baoyuan Wang, Shenghui Cheng, Shuguang Cui, Zhen Li
Abstract
3D single object tracking in LiDAR point clouds (LiDAR SOT) plays a crucial role in autonomous driving. Current approaches all follow the Siamese paradigm based on appearance matching. However, LiDAR point clouds are usually textureless and incomplete, which hinders effective appearance matching. Besides, previous methods greatly overlook the critical motion clues among targets. In this work, beyond 3D Siamese tracking, we introduce a <i>motion-centric paradigm</i> to handle LiDAR SOT from a new perspective. Following this paradigm, we propose a matching-free two-stage tracker <b>M<inline-formula><tex-math notation="LaTeX">$^{2}$</tex-math><alternatives><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msup><mml:mrow/><mml:mn>2</mml:mn></mml:msup></mml:math><inline-graphic xlink:href="zheng-ieq1-3324372.gif" xmlns:xlink="http://www.w3.org/1999/xlink"/></alternatives></inline-formula>-Track</b> . At the 1 <sup>st</sup> -stage, <inline-formula><tex-math notation="LaTeX">$M^{2}$</tex-math></inline-formula> -Track localizes the target within successive frames via <b>m</b> otion transformation. Then it refines the target box through <b>m</b> otion-assisted shape completion at the 2 <sup>nd</sup> -stage. Due to the motion-centric nature, our method shows its impressive generalizability with limited training labels and provides good differentiability for end-to-end cycle training. This inspires us to explore semi-supervised LiDAR SOT by incorporating a pseudo-label-based motion augmentation and a self-supervised loss term. Under the fully-supervised setting, extensive experiments confirm that <inline-formula><tex-math notation="LaTeX">$M^{2}$</tex-math></inline-formula> -Track significantly outperforms previous state-of-the-arts on three large-scale datasets while running at <i>57FPS</i> ( <b><inline-formula><tex-math notation="LaTeX">$\sim$</tex-math><alternatives><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mo>∼</mml:mo></mml:math><inline-graphic xlink:href="zheng-ieq6-3324372.gif" xmlns:xlink="http://www.w3.org/1999/xlink"/></alternatives></inline-formula> 3%</b> , <b><inline-formula><tex-math notation="LaTeX">$\sim$</tex-math><alternatives><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mo>∼</mml:mo></mml:math><inline-graphic xlink:href="zheng-ieq7-3324372.gif" xmlns:xlink="http://www.w3.org/1999/xlink"/></alternatives></inline-formula> 11%</b> and <b><inline-formula><tex-math notation="LaTeX">$\sim$</tex-math><alternatives><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mo>∼</mml:mo></mml:math><inline-graphic xlink:href="zheng-ieq8-3324372.gif" xmlns:xlink="http://www.w3.org/1999/xlink"/></alternatives></inline-formula> 22%</b> precision gains on KITTI, NuScenes, and Waymo Open Dataset respectively). While under the semi-supervised setting, our method performs on par with or even surpasses its fully-supervised counterpart using fewer than half labels from KITTI. Further analysis verifies each component's effectiveness and shows the motion-centric paradigm's promising potential for auto-labeling and unsupervised domain adaptation.