Graph-Transformer with spatial-spectral features fusion for hyperspectral image classification
Zhouzhou Zheng, Mohamed Debbagh, Xuehai Zhou, Shangpeng Sun, Yuxiang Huang
Abstract
Hyperspectral image (HSI) classification plays an important role in interpreting semantics and pixel information. Recently, the graph convolution network (GCN) and vision transformer (ViT) have shown impressive classification capabilities in HSI analysis. Each method offers unique advantages: GCN focuses on local neighborhood features, whereas ViT emphasizes long-range dependencies global features. Existing studies integrated the two methods by serial or parallel for HSI analysis, however, they fell short in deeply fusing the two approaches. To address the challenge, a Graph-Transformer module (GTM) is proposed, which effectively combines local neighborhood features and long-range dependencies global features. Moreover, a spectral feature extraction branch is introduced to enhance spectral learning. Finally, the spatial branch consisting of GTM and spectral branch are fused to complete HSI classification. Experimental results showed that our proposed Graph-Transformer with spatial-spectral features fusion network (GTS 2 F 2 Net) outperformed other state-of-the-art methods on three public datasets. Specifically, it achieved overall accuracy (OA) of 99.31%, 99.69%, and 97.17% on Salinas Valley (SA), Pavia University (PU), Houston 2013, respectively.