Accelerating Neural Style-Transfer Using Contrastive Learning for Unsupervised Satellite Image Super-Resolution
Divya Mishra, Ofer Hadar
Abstract
Contrastive learning is a self-supervised comparison of two samples to identify characteristics and traits that distinguish one data class from another, improving performance on visual tasks. The performance of existing super-resolution-based approaches degrades with increasing scaling factors, hence practically not useful for high-resolution (HR) imaging applications. We proposed a novel framework that uses contrastive training followed by a decoder to generate an “Artificial style image,” which is utilized as a style image for neural style transfer (NST) learning for image super-resolution in an unsupervised manner. The idea is to benefit from HR textures and features as a style and transfer on an original low-resolution (LR) content image as base elements. The proposed framework has three benefits: 1) the framework is capable of super-resolving different modalities of data like single-band remote sensing images, multispectral band images, RGB remote sensing images, and real-world natural images; 2) proposed method outperforms existing unsupervised and also supervised learning-based methods for both visual and qualitative performance; and 3) leveraging NST learning for remote sensing image super-resolution is performed without sacrificing speed and resources. The framework is novel since the work on NST learning to super-resolve remote sensing images in an unsupervised manner has yet to be acknowledged.