CAFE: A Cross-Attention Based Adaptive Weighting Fusion Network for MODIS and Landsat Spatiotemporal Fusion
Liupeng Lin, Yao Shen, Jingan Wu, Fang Nan
Abstract
Dense medium-resolution images play an important role in time-series geoscience applications. However, due to technical limitations, remote sensing imaging systems inevitably trade off temporal frequency and spatial swaths, resulting in difficulties to acquire images simultaneously with high spatial and temporal resolution. To overcome this limitation, under the framework of residual learning, we propose a Cross-attention based Adaptive weighting Fusion nEtwork (CAFE) for MODIS-Landsat spatiotemporal fusion to generate dense medium-resolution images. Based on the cross-attention mechanism, we propose multi-channel separated cross-attention and full-feature joint cross-attention blocks to enhance spatial resolution and retain spectral signatures from the perspectives of band-wise processing and full-feature joint processing, respectively. The adaptive temporal difference weighting mechanism is proposed to improve the ability to capture dynamic land surface changes. Besides, we employ an adaptive fusion loss function to constrain the network training. Experimental results indicate that the developed method is superior to several existing algorithms in terms of visual evaluation and quantitative evaluation and it can generate high-quality fusion results in scenarios of both subtle and dramatic temporal changes. Codes will be available at https://github.com/LiupengLin/CAFE.