DCAFusion: A novel general image fusion framework based on reference image reconstruction and dual-cross attention mechanism
Lixing Fang, Meng Hou, Baoxiang Huang, Ge Chen, Jie Yang
Abstract
In this study, a novel end-to-end image fusion method , DCAFusion, is proposed. The method is based on Swin Transformer and introduces a dual cross-attention mechanism for the task of fusing infrared and visible, multi-focus and medical images. Although infrared and visible datasets contain source image pairs, they lack corresponding labels and cannot be trained by a unified supervised learning framework. To address this problem, DCAFusion designs image reconstruction blocks that generate reconstructed images as labels to guide model feature learning and provide dynamic information retention. The reconstructed images enable the full reference loss function to intervene in a supervised learning manner and participate in the computation of cross-attention scores through information mapping to more efficiently integrate complementary information between source images. In the comparison experiments of infrared and visible image fusion, DCAFusion's fusion result reaches 13.9420 and 1.0895 in the two metrics, ahead of the second-ranked 13.2547 and 0.9983, respectively. These metrics also maintain the lead in comparison experiments of other fusion tasks, which proves DCAFusion's unique advantages in fusion results.