Unsupervised Hyperspectral and Multispectral Image Fusion With Deep Spectral-Spatial Collaborative Constraint
Haoyang Yu, Zhixin Ling, Ke Zheng, Lianru Gao, Jiaxin Li, Jocelyn Chanussot
Abstract
The most cost-effective way to obtain a high spatial resolution hyperspectral image (HrHSI) is to fuse a low spatial resolution hyperspectral image (LrHSI) and corresponding high spatial resolution multispectral image (HrMSI). This article proposes a generalizable unsupervised deep fusion method based on spectral-spatial collaborative constraint to address LrHSI and HrMSI fusion task. First, in view of the limitations of the current spectral-spatial downsampled model, the group convolution enhancement (GCE) module is designed to eliminate the radiometric difference between the images to be fused. Second, to enhance the model’s feature extraction ability, this article introduces the design of the spatial, channel, and filter 3-D attention factor dynamic convolutional kernel (SCFConv). In order to verify the proposed method, we compared and evaluated our method with traditional methods and unsupervised deep learning methods using both simulated and real onboard data, respectively. In the absence of HrHSI validation images in real scenarios, we evaluate the performance of different fusion models through classification results. The experimental results demonstrate the effectiveness of the proposed model and the practical value of the fusion results (the onboard data produced by ours are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://drive.google.com/drive/folders/</uri> 1JLCCB6ld5R49HDLN5SsMISx1d0fuqRjO).