No-Reference Image Quality Assessment via Multibranch Convolutional Neural Networks
Zhaoqing Pan, Feng Yuan, Xu Wang, Long Xu, Xiao Shao, Sam Kwong
Abstract
No-reference image quality assessment (NR-IQA) aims to evaluate image quality without using the original reference images. Since the early NR-IQA methods based on distortion types were only applicable to specific distortion scenarios, and lack of practicality, it is challenging to designing a universal NR-IQA method. In this article, a multibranch convolutional neural network (MB-CNN) based NR-IQA method is proposed, which includes a spatial-domain feature extractor, a gradient-domain feature extractor, and a weight mechanism. The spatial-domain feature extractor aims to extract the distortion features from the spatial domain. The gradient-domain feature extractor is used to guide the spatial-domain feature extractor to pay more attention to the distortions of the structure information. Particularly, the spatial-domain feature extractor uses the hierarchical feature merge module to realize multiscale feature representation, and the gradient-domain feature extractor uses pyramidal convolution to extract the multiscale structure information of the distorted image. In addition, a position vector is proposed to build the weight mechanism by considering the position relationships between patches and its entire image for improving the final prediction performance. We conduct the experiments on five representative databases: LIVE, TID2013, CSIQ, LIVE MD and Waterloo Exploration Database, and the experimental results show that the proposed NR-IQA method achieves the state-of-the-art performance, which demonstrate the effectiveness of our proposed NR-IQA method. The code ofthe proposed MB-CNN will be released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/NUIST-Videocoding/MB-CNN</uri> .