V2VFormer: Vehicle-to-Vehicle Cooperative Perception With Spatial-Channel Transformer
Chunmian Lin, Daxin Tian, Xuting Duan, Jianshan Zhou, Dezong Zhao, Dongpu Cao
Abstract
Collaborative perception aims for a holistic perceptive construction by leveraging complementary information from nearby connected automated vehicle (CAV), thereby endowing the broader probing scope. Nonetheless, how to aggregate individual observation reasonably remains an open problem. In this paper, we propose a novel vehicle-to-vehicle perception framework dubbed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V2VFormer</i> with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Tr</i> ansformer-based <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Co</i> llaboration ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CoTr</i> ). Specifically. it re-calibrates feature importance according to position correlation via Spatial-Aware Transformer ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SAT</i> ), and then performs dynamic semantic interaction with Channel-Wise Transformer ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CWT</i> ). Of note, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CoTr</i> is a light-weight and plug-in-play module that can be adapted seamlessly to the off-the-shelf 3D detectors with an acceptable computational overhead. Additionally, a large-scale cooperative perception dataset V2V-Set is further augmented with a variety of driving conditions, thereby providing extensive knowledge for model pretraining. Qualitative and quantitative experiments demonstrate our proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V2VFormer</i> achieves the state-of-the-art (SOTA) collaboration performance in both simulated and real-world scenarios, outperforming all counterparts by a substantial margin. We expect this would propel the progress of networked autonomous-driving research in the future.