TriTF: A Triplet Transformer Framework Based on Parents and Brother Attention for Hyperspectral Image Change Detection
Xianghai Wang, Keyun Zhao, Xiaoyang Zhao, Siyao Li
Abstract
Hyperspectral image (HSI) change detection (CD) is a technique to accurately detect land cover changes by using HSIs with rich spatial-spectral information. In recent years, the HSI-CD methods based on convolutional neural networks (CNNs) have achieved great success because of their flexible and effective feature extraction ability. However, these methods often take the HSI patches as the input of the networks, which undoubtedly hinders the overall perception of the HSIs. Meanwhile, the valuable temporal information in HSIs is often underutilized. For this end, a triplet transformer framework (TriTF) based on parents-temporal attention and brother-spatial attention is proposed for HSI-CD. The proposed framework mainly contains the following three parts: 1) Transformer-based network backbone, which uses the self-attention to capture the correlation between arbitrarily two pixels in the same patch and extracts the global spatial correlation in the unit of encoded input patches; 2) parents-temporal attention (PTA) branch. Unlike the previous cross-temporal attention mechanisms of the “T1↔T2” mode which only consider the interaction between bi-temporal HSIs, this paper constructs a novel PTA of the “T1→T3←T2” mode which takes the difference-temporal image T3 as the core. The impact of bi-temporal HSIs on the land cover changes is more concerned in the PTA; 3) brother-spatial attention (BSA) branch. The most similar patch in the current training batch of each patch is defined as its brother patch. Furthermore, cross-spatial attention is applied to propagate the features of the brother patch to the current patch. Thus, the middle- and long-range dependencies can be utilized and the scope of feature propagation can be extended. In this paper, the experiments under low and high sampling rates are conducted and proved the outstanding change detection performance of the proposed TriTF when compared with abundant state-of-the-art (SOTA) CD algorithms. The source code of this paper will be released at https://github.com/zkylnnu/TriTF.