Masked Spectral–Spatial Feature Prediction for Hyperspectral Image Classification
Feng Zhou, Chao Xu, Guowei Yang, Renlong Hang, Qingshan Liu
Abstract
Transformer has emerged as a preferred method for hyperspectral (HS) image classification due to its ability to model long-range dependency. Whereas the transformer contains numerous parameters and further available labeled HS data is limited, which makes it difficult to get a well-trained transformer. Accordingly, we propose a novel HS image classification method called masked spectral–spatial feature prediction (MSSFP). It aims at helping the transformer understand the complicated spectral–spatial structures without labeled HS data, further improving the classification performance. Specifically, the input HS cube is first divided into two sequences along spectral and spatial dimensions, respectively. Then, a portion of these two sequences are masked out and we train a transformer-based encoder–decoder network to predict the hand-crafted features of masked regions. After pretraining, the encoder is fine-tuned to derive two classification results from input spectral and spatial sequences. Finally, spectral and spatial results are aggregated adaptively based on uncertainty comparison. In comparison experiments, MSSFP outperforms several state-of-the-art HS image classification methods on three benchmark datasets including Indian Pines (IP), Houston (HU), and Pavia University (PUS).