DSHNet: A Semantic Segmentation Model of Remote Sensing Images Based on Dual Stream Hybrid Network
Yujia Fu, Xiangrong Zhang, Mingyang Wang
Abstract
Semantic segmentation is an important issue in intelligent interpretation of remote sensing, playing an important role in applications such as Earth observation and land data update. However, remote sensing images often contain complex ground objects and the boundaries between them are blurred, which poses a huge challenge to the semantic segmentation task of remote sensing images. This article proposes a Dual Stream Hybrid Network (DSHNet) model, which focuses on parallel extraction of semantic and boundary features in remote sensing images, and improves the performance of semantic segmentation by fully integrating dual stream information. In the Semantic Stream, the ViT model pre-trained on remote sensing images is used as the backbone network for feature extraction. In the Boundary Stream, the boundary detection operator Sobel is used to capture the boundaries of different ground objects in the image, and a Boundary Enhancement Mechanism (BEM) is taken to optimize and enhance the feature representation of ground object boundaries. In addition, DSHNet designs a Feature Fusion Module to cross aggregate features from both semantic and boundary streams. Compared with the state-to-art semantic segmentation methods, DSHNet model has achieved best performance on two datasets of Yellow River Estuary Wetland and GID.