摘要
随着三维探测技术的发展,点云逐渐成为最常用的三维物体/场景表征数据类型之一,广泛应用于自动驾驶、增强现实及虚拟现实等领域。然而,受限于硬件设备、采集环境以及遮挡等因素,采集的原始点云通常是不完整、稀疏、嘈杂的,为点云的处理和分析带来了巨大挑战。在此背景下,点云质量增强技术旨在对原始点云进行处理以获得结构完整、密集且接近无噪的点云,具有重要意义。本文对现阶段深度学习点云质量增强方法进行了系统综述,为后续研究者提供研究基础。首先,简要介绍了点云数据处理中通用的关键技术;分别介绍了补全、上采样和去噪3类点云质量增强方法,并对3类方法中的现有算法进行了分类、梳理及总结。其中,点云补全与点云去噪算法均可根据是否采用编码器—解码器结构分为两大类,点云上采样算法可根据网络主要结构分为基于卷积神经网络的方法、基于生成对抗网络的方法和基于图卷积神经网络的方法。其次,总结了质量增强任务中常用的数据集与评价指标,并分别对比分析了现阶段点云补全、上采样和去噪中主流算法的性能。最后,通过系统的梳理,凝练出点云质量增强方向所面临的挑战,并对未来的研究趋势进行了展望。此外,本文汇总了涉及的文献及其开源代码,详见链接https://github.com/LilydotEE/Point_cloud_quality_enhancement。
With the development of 3D detection technologies,point clouds have gradually become one of the most com⁃mon data representations of 3D objects or scenes that are widely used in many applications,such as autonomous driving,augmented reality(AR),and virtual reality(VR).However,due to limitations in hardware,environment,and occlusion,the acquired point clouds are usually sparse,noisy,and uneven,hence imposing great challenges to the processing and analysis of point clouds.Therefore,point cloud quality enhancement techniques,which aim to process the original point cloud to obtain a dense,clean,and structurally complete point cloud,are of great significance.In recent years,with the development of hardware and machine learning technologies,deep-learning-based point cloud quality enhancement methods,which have great potential to extract the features of point clouds,have attracted the attention of scholars at home and abroad.Related works mainly focus on point cloud completion,point cloud upsampling(also known as super-resolution),and point cloud denoising.Point cloud completion fills the incomplete point clouds to restore the complete point cloud infor⁃mation,while point cloud upsampling increases the point number of the original point cloud to obtain a denser point cloud,and point cloud denoising removes the noisy points in the point cloud to obtain a cleaner point cloud.This paper systemati⁃cally reviews the existing point cloud quality enhancement methods based on deep learning to offer a basis for subsequent research.First,this study briefly introduces the fundamentals and key technologies that are widely used in point cloud analysis.Second,three types of point cloud quality enhancement technologies,namely,upsampling,completion,and denoising,are introduced,classified,and summarized.According to the types of input data,point cloud completion meth⁃ods can be divided into voxel-and point-based algorithms,with the latter being further sub-divided into two types depend⁃ing on whether the encoder-decoder structure is exploited or not.The encoder-decoder-structure-based algorithms can be further divided according to whether the generative adversarial network(GAN)structure is used.Point cloud upsampling methods can be classified into convolutional neural network(CNN)-based algorithms,GAN-based algorithms,and graph convolutional network(GCN)-based algorithms.Point cloud denoising methods can also be divided into two types based on whether the encoder-decoder structure is exploited or not.Third,the commonly used datasets and evaluation metrics in point cloud quality enhancement tasks are summarized.The performance evaluation metrics for geometry reconstruction mainly include chamfer distance,earth mover’s distance,Hausdorff distance,and point-to-surface distance.This paper then compares the state-of-the-art algorithms of point cloud completion and upsampling on common datasets and identifies the reasons for the differences in their performance.The recent progress and challenges in the field are then summarized,and future research trends are proposed.The findings are summarized as follows:1)the point cloud features extracted by existing deep learning-based algorithms are highly global,which means that the local features related to the detailed struc⁃ture cannot be captured well,thus resulting in poor detail reconstruction.Traditional geometric algorithms are known to effectively represent data features based on geometric information.Therefore,how to combine geometric algorithms with deep learning for point cloud quality enhancement is worth exploring.2)Most algorithms are for dense point clouds of single objects,and only a few studies have focused on sparse LiDAR point clouds containing large-scale outdoor scenes.3)Most of the related studies only consider the point cloud processing of a single frame and ignore the temporal correlation of point cloud sequences.Therefore,how to utilize the spatial-temporal correlation to improve quality enhancement perfor⁃mance warrants further investigation.4)In existing methods,the proposed network models are often complex and the infer⁃ence speed is relatively slow,which fail to meet the real-time requirements of several applications.Therefore,how to fur⁃ther reduce the scale of the model parameters and improve the inference speed is a research direction worth exploring.5)Most of the existing methods only process the geometric information(3D coordinates)of point clouds and ignore the attri⁃bute information(e.g.,color and intensity).Therefore,how to simultaneously enhance the quality of geometric and attri⁃bute information needs to be explored.Project page:https://github.com/LilydotEE/Point_cloud_quality_enhancement.
作者
陈建文
赵丽丽
任蓝草
孙卓群
张新峰
马思伟
Chen Jianwen;Zhao Lili;Ren Lancao;Sun Zhuoqun;Zhang Xinfeng;Ma Siwei(School of Information and Communication Engineering,University of Electronic Science and Technology of China,Chengdu 611731,China;China Mobile Research Institute,Beijing 100032,China;School of Computer Science and Technology,University of Chinese Academy of Sciences,Beijing 100190,China;School of Computer Science,Peking University,Beijing 100871,China)
出处
《中国图象图形学报》
CSCD
北大核心
2023年第11期3295-3319,共25页
Journal of Image and Graphics
基金
国家自然科学基金项目(62071449)。
关键词
点云补全
点云上采样
点云去噪
质量增强
深度学习
point cloud completion
point cloud upsampling
point cloud denoising
quality enhancement
deep learning
