Grouped Bidirectional LSTM Network and Multistage Fusion Convolutional Transformer for Hyperspectral Image Classification
Qin Xu, Chao Yang, Jin Tang, Bin Luo
Abstract
The efficiently and effectively discriminative spectral-spatial feature representation is essential for hyperspectral image (HSI) classification. However, most of the existing methods rely on the patch-based convolutional neural networks (CNNs) whose ability of extracting the global spatial information is very limited. To address this issue, in this paper, we propose a two-branch network consisting of a grouped bidirectional long short-term memory (GBiLSTM) network and multi-stage fusion convolutional transformer (MFCT) for HSI classification. In the proposed GBiLSTM-MFCT, to extract the spectral features of HSI efficiently, a GBiLSTM network is designed by dividing the sequence features and hidden units of BiLSTM network into several separate groups. To simultaneously extract the global and local spatial features of HSI, a MFCT is proposed by fusing the features of different levels obtained from the multiple phases of convolutional vision transformer. Moreover, in the multi-headed attention module of each stage, blueprint separable convolution based self-attention (BSCA) module is designed which is able to model the global and local spatial information effectively. The outputs of GBiLSTM network and MFCT are fused to generate discriminative and robust spectral-spatial features for HSI classification. Experiments on three benchmark data sets of IN, UP and KSC demonstrate that the proposed GBiLSTM-MFCT exhibits higher classification performance with very limited labeled samples than eight state-of-the-art methods.