TCCU-Net: Transformer and CNN Collaborative Unmixing Network for Hyperspectral Image
Jianfeng Chen, Chen Yang, Lan Zhang, Linzi Yang, Lifeng Bian, Zijiang Luo, Jihong Wang
Abstract
In recent years, deep learning-based hyperspectral unmixing techniques have garnered increasing attention and made significant advancements. However, relying solely on the use of CNN or Transformer approaches is insufficient for effectively capturing both global and fine-grained information, thereby compromising the accuracy of unmixing tasks. In order to fully harness the information contained within HSIs, this study explores a dual-stream collaborative network, referred to as TCCU-Net. It end-to-end learns information in four dimensions: spectral, spatial, global, and local, to achieve more effective unmixing. The network comprises two core encoders: one is a Transformer encoder, which includes squeeze-launch modules, DSSCR-VIT modules, and stripe pooling modules, while the other one is a CNN encoder, which is composed of 2D pyramid convolutions and 3D pyramid convolutions. By fusing the outputs of these two encoders, the semantic gap between the encoder and decoder is bridged, resulting in improved feature mapping and unmixing outcomes. This study extensively evaluates TCCU-Net and seven hyperspectral unmixing methods on four datasets (Samson, Apex, Jasper Ridge and Synthetic dataset). The experimental results firmly demonstrate that the proposed approach surpasses others in terms of accuracy, holding the potential to effectively address hyperspectral unmixing tasks.