Mutiscale Hybrid Attention Transformer for Remote Sensing Image Pansharpening
Wengang Zhu, Jinjiang Li, Zhiyong An, Zhen Hua
Abstract
Pansharpening methods play a crucial role for remote sensing image processing. The existing pansharpening methods, in general, have the problems of spectral distortion and lack of spatial detail information. To mitigate these problems, we propose a multiscale hybrid attention Transformer pansharpening network (MHATP-Net). In the proposed network, the shallow feature (SF) is first acquired through an SF extraction module (SFEM), which contains the convolutional block attention module (CBAM) and dynamic convolution blocks. The CBAM in this module can filter initial information roughly, and the dynamic convolution blocks can enrich the SF information. Then, the multiscale Transformer module is used to obtain multiencoding feature images. We propose a hybrid attention module (HAM) in the multiscale feature recovery module to effectively address the balance between the spectral feature retention and the spatial feature recovery. In the training process, we use deep semantic statistics matching (D2SM) loss to optimize the output model. We have conducted extensive experiments on several known datasets, and the results show that this article has good performance compared with other state of the art (SOTA) methods.