JTE-CFlow for Low-Light Enhancement and Zero-Element Pixels Restoration With Application to Night Traffic Monitoring Images

Abstract

We observe that the low-light RGB images, as well as night traffic monitoring (NTM) images, contain lots of color pixels with zeros caused by the low-light, which means that the low-light images suffer both information weakness and information loss of zero-element pixels. In this paper, we propose a novel flow-based generative method JTE-CFlow for low-light image enhancement, which consists of a joint-attention transformer based conditional encoder (JTE) and a map-wise cross affine coupling flow (CFlow). Specifically, JTE executes short-range and long-range operations by RRDBs (i.e., residual-in-residual dense blocks) and JATs (i.e., joint-attention transformer blocks) in series connection. JAT achieves weak information amplification and information loss restoration of zero-element pixels by the integration of self-attention and specific-attention with sharing the same value vectors, where the query and key vectors of specific-attention are from the zero-map feature of the low-light image. On the other hand, CFlow develops a map-wise cross affine coupling (MCAC) layer to perform cross learning for the flow feature, and a multiplication coupling network (MCN) to learn the transformation parameters of MCAC. JTE-CFlow learns to map the subtraction of outputs of CFlow and JTE (i.e., the residual code) into a standard normal distribution, and the inverse network of CFlow takes the latent feature of the low-light image as its input to infer the enhanced image. Experiments show that JTE-CFlow outperforms most SOTA methods on 7 mainstream low-light datasets with the same architecture, and can be applied to enhance NTM images. The source code and pre-trained models are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/NJUPT-IPR-HuYin/JTE-CFlow</uri>.

Topics & Concepts