RGTGAN: Reference-Based Gradient-Assisted Texture-Enhancement GAN for Remote Sensing Super-Resolution

Abstract

Reference-based super-resolution (Ref-SR) is a heated topic distinguished from single-image super-resolution (SISR). It aims at transferring more texture details from reference (Ref) image with a different perspective, to super-resolve the low-resolution (LR) image. However, the development of Ref-SR within remote sensing (RS) community is limited by three problems. First, RS images exhibit more complex texture details compared to natural images. It’s challenging to learn and reconstruct fine texture of RS images. Second, the lack of high-quality RS image dataset, which contains massive RS image pairs from different perspectives, hampers the model training and diminishes the generalization of Ref-SR within RS community. Third, the lack of physical system prevents it from verifying the feasibility of Ref-SR in RS practice. To address these problems, this paper proposes a novel reference-based gradient-assisted texture-enhancement GAN (RGTGAN), a novel dataset, namely KaggleSRD, and a novel physical simulation system, namely dual-zoom-lens system (DZLS). Specifically, this paper proposes a gradient-assisted texture-enhancement module (GTEM) to fully release the potential of gradient branch to learn fine structures during feature extraction process, a novel dense-intern deformable convolution (DIDConv) to boost the alignment effect between features from different image branches during feature alignment process, and a novel dense-restore-residual (DRR) module to effectively transfer features. Extensive experimental results on both datasets, RRSSRD and KaggleSRD, demonstrate the superiority of the proposed method over state-of-the-art methods. Furthermore, DZLS verifies promising application prospects of the proposed method. Our code and dataset are publicly available at: https://github.com/stdinR/RGTGAN.

Topics & Concepts