Multiscale Aligned Spatial–Temporal Interaction for Video-Based Person Re-Identification
Zhidan Ran, Xuan Wei, Wei Liu, Xiaobo Lu
Abstract
Video-based person re-identification (Re-ID) aims at retrieving the video clips of the same person across multiple cameras. Since video clips are captured at various spatial resolutions (scales), learning multi-scale person appearance features while constructing the cross-scale information interaction is pivotal for video-based person Re-ID. In this paper, we propose an efficient framework, Multi-Scale Aligned Spatial-Temporal Interaction (MS-STI), which not only exchanges the spatial-temporal information within a scale, but also mines implicit related complementary knowledge across scales. MS-STI presents a hierarchical multi-branch architecture that designs the branches with fewer convolutional layers for lower spatial resolution inputs. In this way, the framework enables inter-scale feature size matching for exchanging information across multiple scale-specific branches. We share the parameters of branched sub-networks to optimize the efficiency of person feature extraction. Furthermore, we propose two modules, Spatial Interaction (SI) and Multi-Scale Temporal Interaction (MSTI), which can realize spatial-temporal interaction across multiple branches. SI performs point-wise spatial information transfer within a frame. While MSTI focuses on inter-frame and inter-scale information interaction. Extensive experiments on three challenging benchmarks demonstrate the effectiveness and superiority of the proposed MS-STI.