FMambaIR: A Hybrid State-Space Model and Frequency Domain for Image Restoration
Xin Luan, Huijie Fan, Qiang Wang, Nan Yang, Shiben Liu, Xiaofeng Li, Yandong Tang
Abstract
With the development of deep learning, impressive progress has been made in the field of image restoration. The existing methods mainly rely on CNN and Transformer to obtain multi-scale feature information. However, these methods rarely integrate frequency domain information effectively during feature extraction, limiting their performance in image restoration. Additionally, few have combined Mamba with the Fourier domain for image restoration, which limits Mamba’s ability to perceive global degradation in the frequency domain. Therefore, we propose a new image restoration model called FMambaIR, which utilizes the complementarity between frequency and Mamba for image restoration. The core of FMambaIR is the F-Mamba block, which combines Fourier transform and Mamba for global degradation perception modeling. Specifically, F-Mamba adopts a dual branch complementary structure, including spatial Mamba branches and Fourier frequency domain global modeling. Mamba models the long-range dependencies of the entire image features, and the frequency branch utilizes Fourier to extract global degraded features from the image. Finally, we use a forward feedback network to integrate local information, which is beneficial for improving the recovery details. We comprehensively evaluate FMambaIR on several image restoration tasks, including underwater image enhancement, remote sensing image dehazing, and low-light image enhancement. The experimental results demonstrate that FMambaIR not only achieves superior performance compared to state-of-the-art methods but also significantly reduces computational complexity. Our code is available at https://github.com/mickoluan/FMambaIR.