SAPNet: Segmentation-Aware Progressive Network for Perceptual Contrastive Deraining
Zheng Shen, Changjie Lu, Yuxiong Wu, Gaurav Gupta
Abstract
Deep learning algorithms have recently achieved promising deraining performance–s on both the natural and synthetic rainy datasets. As an essential low-level preprocessing stage, a deraining network should clear the rain streaks and preserve the fine semantic details. However, most existing methods only consider low-level image restoration. That limits their performances at high-level tasks requiring precise semantic information. To address this issue, in this paper, we present a segmentation aware progressive network (SAPNet) based upon contrastive learning for single image deraining. We start our method with a lightweight derain network formed with progressive dilated units (PDU). The PDU can significantly expand the receptive field and characterize multiscale rain streaks without the heavy computation on multiscale images. A fundamental aspect of this work is an unsupervised background segmentation (UBS) network initialized with ImageNet and Gaussian weights. The UBS can faithfully preserve an image s semantic information and improve the generalization ability to unseen photos. Furthermore, we introduce a perceptual contrastive loss (PCL) and a learned perceptual image similarity loss (LPISL) to regulate model learning. By ex-ploiting the rainy image and ground-truth as the negative and the positive sample in the VGG-16 latent space, we bridge the fine semantic details between the derained image and the ground-truth in a fully constrained manner. Comprehensive experiments on synthetic and real-world rainy images show our model surpasses top-performing methods and aids object detection and semantic segmentation with considerable efficacy. A Pytorch Implementation is available at https://github.com/ShenZheng2000/SAPNetfor-image-deraining.