DBMSTN: A Dual Branch Multiscale Spatio-Temporal Network for dim-small target detection in infrared image
Na Li, Xiangyu Yang, Huijie Zhao
Abstract
Addressing the challenging task of infrared dim and small target (IDST) detection in complex background, which is a major topic in infrared image processing, we propose a Dual Branch Multiscale Spatio-Temporal Network (DBMSTN) to suppress complex background and effectively extract targets’ geometric and motion features. Firstly, DBMSTN utilizes a multiscale spatial feature extraction module that extracts inter-frame difference and saliency feature to highlight small targets at different scales and suppress complex backgrounds. Secondly, the DBMSTN contains a dual-branch spatio-temporal feature extraction module which is designed with improved gating unit in convolutional LSTM (ConvLSTM) to enhance the extraction of motion features to cope with their uncertainty. In addition, DBMSTN achieves a better performance using a fusing module that fuses multilevel spatio-temporal features. It also employs the weighted mean squared error (MSE) loss function with adjustable weights of positive and negative samples to solve the data imbalance problem. Experiments based on two public benchmarks verify that DBMSTN outperforms the state-of-the-art metrics and achieves the highest F1 up to 0.9860, also effectively extracts spatio-temporal features of targets with different speeds. • Multiscale spatio-temporal feature extraction enhances infrared dim-small target detection. • Inter-frame difference and saliency feature highlight small targets and suppress complex background. • Improved ConvLSTM facilitates geometric and motion feature extraction with uncertainty. • The weighted mean squared error loss alleviates the imbalance of positive and negative samples.