Attention-Based Tri-UNet for Remote Sensing Image Pan-Sharpening
Wanwan Zhang, Jinjiang Li, Zhen Hua
Abstract
Pan-sharpening of remote sensing images is a signifi-cant method for integrating remote sensing information in the fieldof computer vision, where complementary and redundant informa-tion between multispectral (MS) images and panchromatic (PAN)images is used to generate high-resolution MS (HRMS) images.Inspired by the remarkable achievements of convolutional neuralnetworks in a variety of computer-vision tasks, we incorporatedomain-specific knowledge to design our attention-based trian-gle UNet (Tri-UNet) architecture to generate high-quality HRMSimages. The attention-based Tri-UNet is mainly divided into thefollowing three modules: 1) feature extraction; 2) feature fusion;and 3) image reconstruction. In the feature extraction step, the fea-ture extraction module simultaneously extracts spectral and spatialinformation from the MS and PAN images. The feature maps arethen fused in the feature fusion module, which makes the finalfeatureimagecontainrichspectralandspatialinformation.Finally,the image reconstruction module generates a high-resolution MSimage that uses the fused image as input. The attention mechanismis introduced into the image reconstruction module to make thenetwork focus more on key information in the feature image. Theexperimental results demonstrate that the proposed method cangenerate high-quality HRMS images. A quantitative comparisonand qualitative analysis of the experimental results indicate thatour method is superior to the existing methods.