MF-Mamba: Multiscale Convolution and Mamba Fusion Model for Semantic Segmentation of Remote Sensing Imagery
Xiao Pu, Yuting Dong, Ji Zhao, Tieqi Peng, Christian Geiß, Yanfei Zhong, Hannes Taubenböck
Abstract
Semantic segmentation of remote sensing imagery plays an important role in applications such as environmental monitoring and disaster response. However, challenges such as complex spatial patterns of variable target objects, significant scale variations, and high inter-class similarity challenge accurate segmentation. Most existing methods based on convolutional neural networks (CNNs) and Transformers face limitations in modeling multi-scale global-local dependencies or often incur high computational costs. Therefore, we propose a multi-scale convolution and mamba fusion model (MF-Mamba) that integrates a CNN encoder with a Mamba-based decoder. The decoder incorporates a Global-Local State Space (GLSS) module with eight-directional selective scanning mechanisms and multi-kernel parallel convolutions to capture the rich global-local context. To enhance multi-scale feature representation, we developed a channel-spatial attention and dense multi-scale feature fusion (CSDF) module, which combines channel-spatial attention and atrous convolutions for multi-scale feature fusion. Additionally, a multi-scale lateral connection is developed to align encoder features for efficient integration. Experiments on the data sets of ISPRS Vaihingen, ISPRS Potsdam, and the Wuhan Dense Labeling Dataset (WHDLD) demonstrate the superior performance of MF-Mamba compared to existing state-of-the-art methods. It achieves Mean F1 scores of 86.71%, 90.70%, and 77.07%, respectively. The code is available at https://github.com/Mango-Mars/MF-Mamba.