MarsFormer: Martian Rock Semantic Segmentation With Transformer
Yonggang Xiong, Xueming Xiao, Meibao Yao, Haiqiang Liu, Hong Yang, Yuegang Fu
Abstract
Semantic segmentation of Mars scenes has a crucial role in Mars rovers science missions. Current CNN-based composition of U-Net has powerful information extraction capabilities; however, convolutional localization suffers from the limited global context modeling capability. Although Transformer global modeling has performed well, it still encounters obstacles in the extraction and retention of low-level features. This issue is particularly relevant for Martian rocks with their varying shapes, textures, and sizes in Mars scenes. In this paper, we propose a novel Transformer semantic segmentation framework for Martian rock images, called MarsFormer, that consists of an encoder-decoder structure connected through a Feature Enhancement Module (FEM) and a Window Transformer Block (WTB). Specifically, multi-scale hierarchical features are generated by the Mix Transformer encoders, Upgraded-FFN decoder (UFD) fuses and filters features at different scales, preserving the rich local and global contextual information. FEM enhances the inter-multi-scale feature correlation from both spatial and channel perspectives. WTB captures the long-range contexts and preserves the local features. We built two datasets of synthetic and real Martian rocks. The synthetic dataset is SynMars, referencing data from the ZhuRong rover taken from its Virtual Terrain Engine. The other dataset is MarsData-V2, from real Mars scenes, and published recently in our previous study. Extensive experiments conducted on both datasets showed that MarsFormer achieves superiority in Martian rock segmentation, obtaining state-of-the-art performance with favorable computational simplicity. The data is available at: https://github.com/CVIR-Lab/SynMars.