Gradient Normalization for Generative Adversarial Networks

Yi-Lun Wu, Hong-Han Shuai, Zhi Rui Tam, Hong-Yu Chiu

20212021 IEEE/CVF International Conference on Computer Vision (ICCV)66 citationsDOI

Abstract

In this paper, we propose a novel normalization method called gradient normalization (GN) to tackle the training instability of Generative Adversarial Networks (GANs) caused by the sharp gradient space. Unlike existing work such as gradient penalty and spectral normalization, the proposed GN only imposes a hard 1-Lipschitz constraint on the discriminator function, which increases the capacity of the discriminator. Moreover, the proposed gradient normalization can be applied to different GAN architectures with little modification. Extensive experiments on four datasets show that GANs trained with gradient normalization outperform existing methods in terms of both Frechet Inception Distance and Inception Score.

Topics & Concepts

DiscriminatorNormalization (sociology)Adversarial systemComputer scienceArtificial intelligenceGenerative grammarGenerative adversarial networkPattern recognition (psychology)AlgorithmDeep learningTelecommunicationsDetectorAnthropologySociologyGenerative Adversarial Networks and Image SynthesisAnomaly Detection Techniques and ApplicationsAdvanced Image Processing Techniques