Cross-Modal Local Calibration and Global Context Modeling Network for RGB–Infrared Remote-Sensing Object Detection
Jin Xie, Jing Nie, Bonan Ding, Mingyang Yu, Jiale Cao
Abstract
RGB-infrared object detection in remote sensing images is crucial for achieving around-clock surveillance of unmanned aerial vehicles. RGB-infrared remote sensing object detection methods based on deep learning usually mine the complementary information from RGB and infrared modalities by utilizing feature aggregation to achieve robust object detection for around-the-clock applications. Most of the existing methods aggregate features from RGB and infrared images by utilizing element-wise operations ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">eg.,</i> element-wise addition or concatenation). The detection accuracy of these methods is limited. The main reasons can be concluded as follows: local location misalignment across modalities and insufficient non-local contextual information extraction. To address the above issues, we propose a cross-modal local calibration and global context modeling network (CLGNet), consisting of two novel modules: a cross-modal local calibration (CLC) module and a cross-modal global context modeling (CGC) module. The CLC module first aligns features from different modalities, and then aggregates them selectively. The CGC module is embedded into the backbone network to capture cross-modal non-local long-range dependencies. Experimental results on popular RGB-infrared remote sensing object detection datasets, namely DRoneVehicle and VEDAI demonstrate the effectiveness and efficiency of our CLGNet.