Crossed Dual-Branch U-Net for Hyperspectral Image Super-Resolution
Jingyi Zhang, Jianjun Liu, Jinlong Yang, Zebin Wu
Abstract
Hyperspectral images have gained great achievements in many fields, but their low-spatial resolution limits the effectiveness in applications. Hyperspectral image super-resolution has emerged as a popular research trend, where high-resolution hyperspectral images are obtained via combing low-resolution hyperspectral images with high-resolution multispectral images. In this process of multi-modality data fusion, it is crucial to ensure effective cross-modality information interaction. To generate higher quality fusion results, a crossed dual-branch U-Net is proposed in this paper. In specific, we adopt U-Net architecture and introduce a spectral-spatial feature interaction module to capture cross-modality interaction information between two input images. To narrow the gap between downsampling and upsampling processes, a spectral-spatial parallel Transformer is designed as skip connection. This novel design simultaneously learns the long-range dependencies both on spatial and spectral information and provides detailed information for final fusion. In the fusion stage, We adopt a progressive upsampling strategy to refine the generated images. Extensive experiments on several public datasets are conducted to prove the performance of the proposed network. The code can be accessed at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/liuofficial/CD-UNet</uri> .