A Novel Cross-Scale Octave Network for Hyperspectral and Multispectral Image Fusion
Tianming Zhan, Zuolin Bi, Huapeng Wu, Chao Xu, Qian Du, Yang Xu, Zebin Wu
Abstract
Recently, deep convolutional neural network-based low-resolution hyperspectral image (LR-HSI) and high-resolution multispectral image (HR-MSI) fusion methods have achieved significant performance improvement. However, the rich spatial and spectral information in HSIs is not fully explored. In this article, we propose a novel cross-scale octave network (CSONet) for hyperspectral and multispectral image fusion. Specifically, we adopt a progressive image fusion structure to effectively extract the spatial and spectral information of HR-MSI at multiple resolutions, thereby efficiently complementing LR-HSI’s information. In addition, the proposed cross-scale octave convolution module can extract rich multiscale spatial feature information and concentrate on more important spatial–spectral features at different scales with the multiscale spatial–spectral attention mechanism. Finally, a multisupervised loss function is used to improve the gradient propagation and enhance the representation ability of the network. Ablation analysis on the benchmark datasets shows the effectiveness of each component in the proposed method. Extensive experimental results on different hyperspectral images demonstrate that the proposed CSONet can achieve superior results and strong generalization ability in comparison with some state-of-the-art LR-HSI and HR-MSI fusion methods.