Cascaded Memory Network for Optical Remote Sensing Imagery Cloud Removal

Abstract

Cloud removal is an inevitable task in optical remote sensing images, which aims at restoring high-quality images from cloud-contaminated images. In recent years, deep learning-based image cloud removal methods utilize convolution neural network to obtain clean images. However, due to the limitations of the convolution operator, these methods cannot effectively leverage the local and global information of the image. In this paper, we propose a Cascaded Memory Network (CMNet) for optical remote sensing imagery cloud removal. The CMNet recycles previously captured information to form a memory mechanism, which is composed of two cascaded components: local information memory module (LIMM) and global information auxiliary module (GIAM). The LIMM aims to obtain local spatial details information of the image via two sub-networks, and the GIAM tries to further restore the detail of the image from global perspective. In the LIMM, two sub-networks are constructed to capture the details from coarse to fine, each of which includes a continuous memory descriptor that describes local details of the image and a hierarchical memory correlation descriptor that adaptively integrates relevant features. In the GIAM, we design a swin cloud remove transformer layer and explore an adaptive normalization to cope with unevenly distributed thin clouds, and further provide theoretical proof for the existence of the required solution. Experimental results indicate that our method can remove clouds while maintaining the detailed information of the image. https://github.com/Lab-PANbin/.

Topics & Concepts