Convolutional Neural Network for 3D Point Cloud Quality Assessment with Reference
Aladine Chetouani, Maurice Quach, Giuseppe Valenzise, Fréderic Dufaux
Abstract
In recent years, the production of 3D content in the form of point clouds (PC) has increased considerably, especially in virtual reality applications. This enthusiasm is linked in particular to the development of acquisition technologies. In order to ensure a good quality of user experience, it is necessary to offer a high quality of visualization whatever the transmission medium used or the treatments applied. Thus, several metrics have been proposed which are essentially point-based metrics. In this article, we propose a deep learning-based method that efficiently predicts the quality of distorted PCs thanks to a set of features extracted from selected patches of the reference PC and its degraded version as well as the use of Convolutional Neural Networks (CNNs). The patches are selected randomly and the difference between corresponding patches is characterized by three attributes: geometry, curvature and color. The proposed method was evaluated and compared to state-of-the-art metrics using two datasets, including a large dataset more suited to deep learning models. We also compared different symmetrization functions and machine learning pooling as well as the ability of our method to predict the quality of unknown PCs through a cross-dataset evaluation. The results obtained show the relevance of the proposed framework with interesting perspectives.