<i>UNFOLD</i>: 3-D U-Net, 3-D CNN, and 3-D Transformer-Based Hyperspectral Image Denoising
Aditya Dixit, Anup Kumar Gupta, Puneet Gupta, Saurabh Srivastava, Ankur Garg
Abstract
Hyperspectral Images (HSIs) encompass data across numerous spectral bands, making them valuable in various practical fields such as remote sensing, agriculture, and marine monitoring. Unfortunately, inevitable noise introduction during sensing restricts their applicability, necessitating denoising for optimal utilization. The existing Deep Learning based denoising methods suffer from various limitations. For instance, Convolutional Neural Networks (CNNs) struggle with long-range dependencies, while Vision Transformers struggle to capture local details. This paper introduces a novel method, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">UNFOLD</i> , that addresses these inherent limitations by harmoniously integrating the strengths of 3D U-Net, 3D CNN, and 3D Transformer architectures. Unlike several existing methods that predominantly capture dependencies either along the spatial or the spectral dimension, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">UNFOLD</i> addresses HSI denoising as a 3D task, synergizing spatial and spectral information through the utilization of 3D Transformer and 3D CNN. It employs the self-attention mechanism of Transformers to capture the global dependencies and model long-range relationships across spatial and spectral dimensions. To overcome the limitations of 3D Transformer in capturing fine-grained local and spatial features, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">UNFOLD</i> complements it by incorporating 3D CNN. Moreover, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">UNFOLD</i> utilize a modified form of 3D U-Net architecture for HSI denoising, wherein it employs a 3D Transformer based encoder instead of the conventional 3D CNN-based encoder. It further capitalizes on the property of U-Net to integrate features across various scales, thereby enhancing efficacy by preserving intricate structural details. Results from extensive experiments demonstrate that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">UNFOLD</i> outperforms the state-of-the-art HSI denoising methods.