Pansharpening Method Based on Deep Nonlocal Unfolding
Xingxing Li, Yujia Li, Guangyao Shi, Liping Zhang, Weisheng Li, Dajiang Lei
Abstract
Although deep neural networks (DNNs) have achieved great success in pansharpening, most of them lack transparency and interpretability. Currently, some DNNs methods utilize deep unfolding techniques to alleviate this problem. However, they do not consider the regularization term separately when solving the energy function that represents the image degradation process, making it difficult to extract complex prior information in the unfolding module. Therefore, this paper proposes a pansharpening method based on deep non-local unfolding. Specifically, we expand the iterative process of solving the energy function into the corresponding neural network modules, making each module have a certain physical meaning. Then, we decouple the prior operator containing the prior knowledge of the remote sensing image and approximate the solution using the network module. Meanwhile, we incorporate local and non-local self-similarity priors into the prior operator and design a two-branch prior module for learning the prior features and contribution weights adaptively. Finally, the fused image is corrected with the learned prior features to approximate the real image. Experimental results on datasets from two different types of satellites demonstrate the superiority of our approach.