Latent Denoising Diffusion GAN: Faster Sampling, Higher Image Quality
Luan Thanh Trinh, Tomoki Hamagami
Abstract
Diffusion models are emerging as a powerful solution for generating high-fidelity and diverse images, often surpassing GANs under many circumstances. However, their slow inference speeds hinder their potential for real-time applications. To address this, DiffusionGAN leveraged a conditional GAN to drastically reduce denoising steps and speed up inference. Its advancement, Wavelet Diffusion, further accelerated the process by converting data into wavelet space, thus enhancing efficiency. Nonetheless, these models still fall short of GANs in terms of speed and image quality. To bridge these gaps, this paper introduces the Latent Denoising Diffusion GAN, which employs pre-trained autoencoders to compress images into a compact latent space, significantly improving inference speed and image quality. Furthermore, we propose a Weighted Learning strategy to enhance diversity and image quality. Experimental results on the CIFAR-10, CelebA-HQ, and LSUN-Church datasets prove that our model achieves a state-of-the-art running speed among diffusion models. Compared to its predecessors, DiffusionGAN and Wavelet Diffusion, our model shows remarkable improvements on all evaluation metrics. Code and pre-trained checkpoints: https://github.com/thanhluantrinh/LDDGAN.git.