Hyperspectral Image Instance Segmentation Using Spectral–Spatial Feature Pyramid Network
Leyuan Fang, Yifan Jiang, Yinglong Yan, Jun Yue, Yue Deng
Abstract
In recent years, hyperspectral image (HSI) classification and detection techniques based on deep learning have been widely applied to various aspects, such as environmental monitoring, urban planning, and energy surveys. As an important image content analysis method, instance segmentation can provide important support for the extraction of ground object information and monomeric application of HSI. This article introduces instance segmentation into HSI interpretation for the first time. In this article, we create the hyperspectral instance segmentation dataset (HS-ISD), which contains a total of 56 images, each with a size of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$298\times301$ </tex-math></inline-formula> and a number of channels of 48. More than 1000 architectural examples are annotated to apply to the research of HSI instance segmentation. In addition, considering that HSI contains rich spectral and spatial information, and the traditional instance segmentation network model cannot well utilize both types of information effectively, we propose the spectral–spatial feature pyramid network (Spectral–Spatial FPN). The Spectral–Spatial FPN can integrate multiscale spectral information and multiscale spatial information in the feature extraction stage through attention mechanism and bidirectional feature pyramid structure, so as to better improve the performance of the network model by spectral information and spatial information and realize the end-to-end instance segmentation of HSI. The experimental results conducted on the HS-ISD show that the proposed Spectral–Spatial FPN can achieve state-of-the-art results.