Attention_FPNet: Two-Branch Remote Sensing Image Pansharpening Network Based on Attention Feature Fusion
Xiwu Zhong, Yurong Qian, Hui Liu, Long Chen, Yaling Wan, Liang Gao, Jing Qian, Jun Liu
Abstract
Inspired by the impressive achievements of convolutional neural networks (CNNs) in various computer vision tasks and the effective role of attention mechanisms, this paper proposes a two-branch fusion network based on attention feature fusion (AFF) called Attention FPNet to solve the pansharpening problem. We reconstruct the spatial information of an image in the high-pass filter domain and fully consider the spatial information in the multispectral (MS) and panchromatic (PAN) images. At the same time, the input PAN image and the upsampled MS image are directly transmitted to the reconstructed image through a long skip connection. The spectral information of the PAN and MS images is considered to improve the spectral resolution of the fused image. It also supplements the loss of spatial information that may be caused by network deepening. Moreover, an AFF method is used to replace the existing simple channel concatenation method commonly used in pansharpening, which fully considers the relationship between different feature maps and improves the fusion quality. Through experiments on image datasets acquired by the Pleiades, SPOT-6 and Gaofen-2 satellites, the results show that this method can effectively fuse PAN and MS images and generate a fused image and outperforms existing methods.