MTDFusion: A Multilayer Triple Dense Network for Infrared and Visible Image Fusion
Shahid Karim, Geng Tong, Jinyang Li, Xiaochang Yu, Jia Hao, Akeel Qadir, Yiting Yu
Abstract
The ultimate image with important information and better visuals can be obtained by fusing infrared and visual images. Several fusion methods consist of complex networks to derive the finest parameters from different source images and do not exploit reliable and salient information completely. To avoid the limitations of information loss and fusion of feature information, we have proposed a multilayer triple dense network for infrared and visible image fusion named as MTDFusion, which completely exploits salient features and utilizes the residual features from the combination of input images adaptively. The encoding network is designed as a triple network of dense blocks that takes inputs as infrared, visible, and the difference extracted from infrared and visible images. The fusion layer exploits the weighted combination of input images, and decoding layers work conventionally. The results obtained by our proposed approach reveal optimal image details and higher contrast. We have improved the loss function by combining smooth-ℒ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> and structured similarity index (SSIM) loss functions for our proposed method. Additionally, the different variations of λs in SSIM are examined using qualitative and quantitative assessments. The results achieve optimal fusion performance compared to several state-of-the-art methods. The source code for MTDFusion is available at https://github.com/tgg-77/MTDFusion.