Spectral–Spatial Adversarial Multidomain Synthesis Network for Cross-Scene Hyperspectral Image Classification
Xi Chen, Lin Gao, Maojun Zhang, Chen Chen, Shen Yan
Abstract
Cross-scene hyperspectral image (HSI) classification has received widespread attention due to its practicality. However, domain adaptation-based cross-scene HSI classification methods are typically tailored for a specific target scene involved in model training and require retraining for new scenes. We instead propose an novel spectral-spatial adversarial multi-domain synthetic network (S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> AMSnet) that can be trained on a single source domain (SD) and generalized to unseen domains. S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> AMSnet improves the robustness of the model to the unseen domain by expanding the diverse distribution of the SD. Specifically, to spatially and spectrally generate diversified generative domain (GD), the spectral-spatial domain generation network (S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> DGN) is designed, and two S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> DGNs with the same structure but not shared parameters are enabled to generate diversified GD through two-step min-max strategy. A Multi-domain mixing module is employed to expand the diversity of the GD further and enhance their class-domain semantic consistency information. Additionally, a multi-scale mutual information regularization network is used to constrain the S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> DGN so that the intrinsic class semantic information of its generated GD does not deviate from the SD. A Semantic consistency discriminator with spectral-spatial feature extraction capability is utilized to capture class-domain semantic consistency information from diverse GD to obtain cross-domain invariant knowledge. Comparative analysis with eight state-of-the-art transfer learning methods on three real HSI datasets, along with an ablation study, validates the effectiveness of the proposed S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> AMSnet in the cross-scene HSI classification task. The codes of this work will be available at https://github.com/daxichen/S2AMSnet.