Integrating Multisubspace Joint Learning With Multilevel Guidance for Cross-Modal Retrieval of Remote Sensing Images
Yaxiong Chen, Jirui Huang, Shengwu Xiong, Xiaoqiang Lu
Abstract
In recent years, with the continuous advancement of remote sensing technology and text processing techniques, there has been a growing abundance of remote sensing images and associated textual data. Combining remote sensing images with their corresponding textual data allows for integrated analysis and retrieval, which holds significant practical implications across multiple application domains, including geographic information systems (GIS), environmental monitoring, and agricultural management. Remote sensing images have the characteristics of multi-targets and multi-scales, and the textual descriptions of these targets are not fully utilized, leading to a decrease in retrieval accuracy. Previous methods have struggled to balance inter-modality information interaction and intra-modality feature fusion, and they have paid little attention to the consistency of distribution within modalities. In light of this, this paper proposes a symmetric multi-level guidance network (SMLGN) for cross-modal retrieval in remote sensing. SMLGN first introduces fusion guidance between local and global within modalities and fine-grained bidirectional guidance between modalities, allowing for the learning of a common semantic space. Furthermore, to address the distribution differences of different modalities within the common semantic space, we design an adversarial joint learning framework and a multi-objective loss function to optimize the SMLGN method and achieve consistency in data distribution. The experimental results demonstrate that the SMLGN method performs well in the task of cross-modal retrieval between remote sensing images and textual data. It effectively integrates the information from both modalities, improving the accuracy and reliability of the retrieval process.