MobileAR-GAN: MobileNet-Based Efficient Attentive Recurrent Generative Adversarial Network for Infrared-to-Visual Transformations
Nand Kumar Yadav, Satish Kumar Singh, Shiv Ram Dubey
Abstract
Deep learning has recently shown outstanding performance for different applications, including image-to-image translation by generative adversarial networks (GANs). However, GAN models are very complex as build with multiple deep networks and require huge computational resources for the training and inference. Hence, the real-time deployment of GAN models is not feasible at present. In this article, we propose MobileNet-based efficient attentive recurrent GAN (MobileAR-GAN) for resource-constrained infrared-to-visual translation. The proposed model utilizes the lightweight MobileNet and enhances its capacity using the attention and recurrent modules, leading to an efficient yet effective model. We consider the infrared to visible image translation task to validate the efficiency and performance of the proposed model. The proposed MobileAR-GAN outperforms most of the existing GAN models in terms of both the efficiency and the quality of the generated images. We also test the MobileAR-GAN model over the resource-limited Jetson TX2 board with very compelling results. The proposed model shows promising results over state-of-the-art methods. Compared to lightweight models, such as Pix2pix and GAN-Compression methods, an average improvement gain of 19.19% and 17.05% is observed by the proposed model in terms of SSIM metric. It is observed that the proposed model can be deployed on edge devices to transform the images taken at night time using an infrared camera to the corresponding visible images with satisfactory performance.