CR-DINO: A Novel Camera-Radar Fusion 2-D Object Detection Model Based on Transformer
Yuhao Jin, Xiaohui Zhu, Yong Yue, Eng Gee Lim, Wei Wang
Abstract
Due to millimeter-wave (MMW) radar’s ability to directly acquire spatial positions and velocity information of objects, as well as its robust performance in adverse weather conditions, it has been widely employed in autonomous driving. However, radar lacks specific semantic information. To address this limitation, we take the complementary strengths of camera and radar by feature-level fusion and propose a fully transformer-based model for object detection in autonomous driving. Specifically, we introduce a novel radar representation method and propose two camera-radar fusion architectures based on Swin transformer. We name our proposed model as camera-radar based DETR with improved denoising anchor boxes (CR-DINO) and conduct training and testing on the nuScenes dataset. We conducted several ablation experiments, and the best result we obtained was an mAP of 38.0%, surpassing other state-of-the-art (SOTA) camera-radar fusion object detection models.