RegR-PCQA: Deep Learning Based Colored Point Cloud Quality Assessment Using 3D-to-2D Regularized Representation

Point Cloud Quality Assessment (PCQA) aims to accurately predict the visual quality of a point cloud, which is essential in optimizing and evaluating the point cloud compression, transmission and rendering. In this paper, we propose a deep learning based full reference PCQA using 3D-to-2D Regularized Representation (RegR-PCQA), where point clouds are projected to regularized 2D image representations and then measured with deep neural networks. Firstly, we propose a regularized representation module to project unstructured point clouds to 2D Regularized Geometry Images (RGIs) and Regularized Attribute Images (RAIs), which enhance the local adjacency and uniform distribution of points. An anchor matching is developed to build the correspondence of regularized images between the distorted and reference point clouds. Secondly, to exploit the visual features of the RGIs and RAIs, we propose a deep learning based two-branch PCQA network, in which vision transformer based Geometry Feature Extractor (GFE) extracts global structural features from RGIs and Convolutional Neural Network (CNN) based Attribute Feature Extractor (AFE) extracts local semantic features of the RAIs. Finally, based on the geometry and attribute features, the point cloud quality is predicted by the proposed quality regression module, where a spatial attention mechanism is exploited to assign different importance weights for the feature maps. Experimental results show that the Pearson Linear Correlation Coefficients (PLCC) achieved by the proposed RegR-PCQA are 0.8430, 0.9575, 0.7853 and 0.8576, respectively, on the SIAT-PCQD, SJTU-PCQA, WPC and WPC2.0 datasets, which are superior to the state-of-the-art PCQAs. Also, extensive experimental results on distortion types, sampling strategy and training rate show that the proposed RegR-PCQA achieves an excellent generalization.