Cross-Channel Dynamic Spatial–Spectral Fusion Transformer for Hyperspectral Image Classification
Zhao Qiu, Jie Xu, Jiangtao Peng, Weiwei Sun
Abstract
Convolutional neural network (CNN) has achieved great success in hyperspectral image (HSI) classification. However, the local receptive field of CNN leads to the drawback in extracting long-distance features. Transformer has excellent global modeling ability and shows good performance for HSI classification. The existing Transformer-based methods usually ignore a problem that the spatial information varies under different channels. To well describe the cross-channel dependencies, a cross-channel dynamic spatial-spectral fusion transformer (CDSFT) is proposed in this article. In the proposed CDSFT, the multi-scale and multi-channel features are extracted and then cross-channel global features are extracted through transpose multi-head self-attention (TMHSA). Next, a dynamic feature enhancement module and a spectral spatial position attention module are designed to extract and enhance spectral-spatial joint features for classification. Experimental results on three well-known HSI datasets demonstrate the effectiveness of the proposed CDSFT method.