PCDNF: Revisiting Learning-based Point Cloud Denoising via Joint Normal Filtering (2209.00798v2)
Abstract: Recovering high quality surfaces from noisy point clouds, known as point cloud denoising, is a fundamental yet challenging problem in geometry processing. Most of the existing methods either directly denoise the noisy input or filter raw normals followed by updating point positions. Motivated by the essential interplay between point cloud denoising and normal filtering, we revisit point cloud denoising from a multitask perspective, and propose an end-to-end network, named PCDNF, to denoise point clouds via joint normal filtering. In particular, we introduce an auxiliary normal filtering task to help the overall network remove noise more effectively while preserving geometric features more accurately. In addition to the overall architecture, our network has two novel modules. On one hand, to improve noise removal performance, we design a shape-aware selector to construct the latent tangent space representation of the specific point by comprehensively considering the learned point and normal features and geometry priors. On the other hand, point features are more suitable for describing geometric details, and normal features are more conducive for representing geometric structures (e.g., sharp edges and corners). Combining point and normal features allows us to overcome their weaknesses. Thus, we design a feature refinement module to fuse point and normal features for better recovering geometric information. Extensive evaluations, comparisons, and ablation studies demonstrate that the proposed method outperforms state-of-the-arts for both point cloud denoising and normal filtering.
- M. Wei, H. Chen, Y. Zhang, H. Xie, Y. Guo, and J. Wang, “GeoDualCNN: Geometry-supporting dual convolutional neural network for noisy point clouds,” IEEE Trans. Vis. Comput. Graph., 2021.
- H. Chen, Z. Wei, X. Li, Y. Xu, M. Wei, and J. Wang, “RePCD-Net: Feature-aware recurrent point cloud denoising network,” Int. J. Comput. Vision, vol. 130, no. 3, pp. 615–629, 2022.
- D. Zhang, X. Lu, H. Qin, and Y. He, “Pointfilter: Point cloud filtering via encoder-decoder modeling,” IEEE Trans. Vis. Comput. Graph., vol. 27, no. 3, pp. 2015–2027, 2021.
- A. C. Öztireli, G. Guennebaud, and M. Gross, “Feature preserving point set surfaces based on non-linear kernel regression,” Comput. Graph. Forum, vol. 28, no. 2, pp. 493–501, 2009.
- X. Lu, S. Wu, H. Chen, S.-K. Yeung, W. Chen, and M. Zwicker, “GPF: GMM-inspired feature-preserving point set filtering,” IEEE Trans. Vis. Comput. Graph., vol. 24, no. 8, pp. 2315–2326, 2017.
- H. Chen, M. Wei, Y. Sun, X. Xie, and J. Wang, “Multi-patch collaborative point cloud denoising via low-rank recovery with graph constraint,” IEEE Trans. Vis. Comput. Graph., vol. 26, no. 11, pp. 3255–3270, 2019.
- H. Chen, J. Huang, O. Remil, H. Xie, J. Qin, Y. Guo, M. Wei, and J. Wang, “Structure-guided shape-preserving mesh texture smoothing via joint low-rank matrix recovery,” Comput-Aided Des., vol. 115, pp. 122–134, 2019.
- X. Lu, S. Schaefer, J. Luo, L. Ma, and Y. He, “Low rank matrix approximation for 3d geometry filtering,” IEEE Trans. Vis. Comput. Graph., vol. 28, no. 4, pp. 1835–1847, 2020.
- Z. Liu, X. Xiao, S. Zhong, W. Wang, Y. Li, L. Zhang, and Z. Xie, “A feature-preserving framework for point cloud denoising,” Comput-Aided Des., vol. 127, p. 102857, 2020.
- Z. Liu, Y. Li, W. Wang, L. Liu, and R. Chen, “Mesh total generalized variation for denoising,” IEEE Trans. Vis. Comput. Graph., vol. 28, no. 12, pp. 4418–4433, 2022.
- M.-J. Rakotosaona, V. La Barbera, P. Guerrero, N. J. Mitra, and M. Ovsjanikov, “PointCleanNet: Learning to denoise and remove outliers from dense point clouds,” Comput. Graph. Forum, vol. 39, no. 1, pp. 185–203, 2020.
- D. Lu, X. Lu, Y. Sun, and J. Wang, “Deep feature-preserving normal estimation for point cloud filtering,” Comput-Aided Des., vol. 125, p. 102860, 2020.
- Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic graph cnn for learning on point clouds,” ACM Trans. Graph., vol. 38, no. 5, pp. 1–12, 2019.
- L. Zhou, G. Sun, Y. Li, W. Li, and Z. Su, “Point cloud denoising review: from classical to deep learning-based approaches,” Graph. Models, vol. 121, p. 101140, 2022.
- M. Alexa, J. Behr, D. Cohen-Or, S. Fleishman, D. Levin, and C. T. Silva, “Computing and rendering point set surfaces,” IEEE Trans. Vis. Comput. Graph., vol. 9, no. 1, pp. 3–15, 2003.
- N. Amenta and Y. J. Kil, “Defining point-set surfaces,” ACM Trans. Graph., vol. 23, no. 3, pp. 264–270, 2004.
- S. Fleishman, D. Cohen-Or, and C. T. Silva, “Robust moving least-squares fitting with sharp features,” ACM Trans. Graph., vol. 24, no. 3, pp. 544–552, 2005.
- Y. Lipman, D. Cohen-Or, D. Levin, and H. Tal-Ezer, “Parameterization-free projection for geometry reconstruction,” ACM Trans. Graph., vol. 26, no. 3, p. 22, 2007.
- H. Huang, D. Li, H. Zhang, U. Ascher, and D. Cohen-Or, “Consolidation of unorganized point clouds for surface reconstruction,” ACM Trans. Graph., vol. 28, no. 5, pp. 176:1–7, 2009.
- H. Huang, S. Wu, M. Gong, D. Cohen-Or, U. Ascher, and H. R. Zhang, “Edge-aware point set resampling,” ACM Trans. Graph., vol. 32, no. 1, pp. 9:1–9:12, 2013.
- R. Preiner, O. Mattausch, M. Arikan, R. Pajarola, and M. Wimmer, “Continuous projection for fast L1subscriptL1\textsc{L}_{1}L start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT reconstruction.” ACM Trans. Graph., vol. 33, no. 4, pp. 47–1, 2014.
- H. Avron, A. Sharf, C. Greif, and D. Cohen-Or, “ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-sparse reconstruction of sharp point set surfaces,” ACM Trans. Graph., vol. 29, no. 5, pp. 135:1–12, 2010.
- Y. Sun, S. Schaefer, and W. Wang, “Denoising point sets via ℓ0subscriptℓ0\ell_{0}roman_ℓ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT minimization,” Comput. Aided Geom. Des., vol. 35, pp. 2–15, 2015.
- E. Mattei and A. Castrodad, “Point cloud denoising via moving RPCARPCA\mathrm{RPCA}roman_RPCA,” Comput. Graph. Forum, vol. 36, no. 8, pp. 123–137, 2017.
- J. Digne, S. Valette, and R. Chaine, “Sparse geometric representation through local shape probing,” IEEE Trans. Vis. Comput. Graph., vol. 24, no. 7, pp. 2238–2250, 2017.
- J. Wang, J. Jiang, X. Lu, and M. Wang, “Rethinking point cloud filtering: A non-local position based approach,” Comput-Aided Des., vol. 144, p. 103162, 2022.
- G. Sun, C. Chu, J. Mei, W. Li, and Z. Su, “Structure-aware denoising for real-world noisy point clouds with complex structures,” Comput-Aided Des., vol. 149, p. 103275, 2022.
- W. Hu, X. Gao, G. Cheung, and Z. Guo, “Feature graph learning for 3d point cloud denoising,” IEEE Trans. Signal Process., vol. 68, pp. 2841–2856, 2020.
- C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “PointNet: Deep learning on point sets for 3d classification and segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 652–660.
- C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “PointNet++: Deep hierarchical feature learning on point sets in a metric space,” in Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS), 2017, pp. 5100–5109.
- S.-L. Liu, H.-X. Guo, H. Pan, P.-S. Wang, X. Tong, and Y. Liu, “Deep implicit moving least-squares functions for 3d reconstruction,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 1788–1797.
- D. de Silva Edirimuni, X. Lu, Z. Shao, G. Li, A. Robles-Kelly, and Y. He, “IterativePFN: True iterative point cloud filtering,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
- R. Roveri, A. C. Öztireli, I. Pandele, and M. Gross, “PointProNets: Consolidation of point clouds with convolutional neural networks,” Comput. Graph. Forum, vol. 37, no. 2, pp. 87–99, 2018.
- L. Yu, X. Li, C.-W. Fu, D. Cohen-Or, and P.-A. Heng, “Ec-net: an edge-aware point set consolidation network,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 386–402.
- S. Luo and W. Hu, “Differentiable manifold reconstruction for point cloud denoising,” in Proceedings of the ACM International Conference on Multimedia (ACM MM), 2020, pp. 1330–1338.
- P. Hermosilla, T. Ritschel, and T. Ropinski, “Total denoising: Unsupervised learning of 3d point cloud cleaning,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019, pp. 52–60.
- F. Pistilli, G. Fracastoro, D. Valsesia, and E. Magli, “Learning graph-convolutional representations for point cloud denoising,” in Proceedings of the European Conference on Computer Vision (ECCV), 2020, pp. 103–118.
- S. Luo and W. Hu, “Score-based point cloud denoising,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021, pp. 4583–4592.
- H. Chen, B. Du, S. Luo, and W. Hu, “Deep point set resampling via gradient fields,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 3, pp. 2913–2930, 2023.
- D. de Silva Edirimuni, X. Lu, G. Li, and A. Robles-Kelly, “Contrastive learning for joint normal estimation and point cloud filtering,” IEEE Trans. Vis. Comput. Graph., 2023.
- M. Kazhdan and H. Hoppe, “Screened poisson surface reconstruction,” ACM Trans. Graph., vol. 32, no. 3, pp. 1–13, 2013.
- F. Hou, C. Wang, W. Wang, H. Qin, C. Qian, and Y. He, “Iterative Poisson surface reconstruction (iPSR) for unoriented points,” ACM Trans. Graph., vol. 41, no. 4, pp. 128:1–13, 2022.
- R. B. Rusu, N. Blodow, and M. Beetz, “Fast point feature histograms (FPFH) for 3d registration,” in IEEE International Conference on Robotics and Automation (ICRA), 2009, pp. 3212–3217.
- J. Zhang, Y. Yao, and B. Deng, “Fast and robust iterative closest point,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 7, 2022.
- P. Guerrero, Y. Kleiman, M. Ovsjanikov, and N. J. Mitra, “PCPNet learning local shape properties from raw point clouds,” Comput. Graph. Forum, vol. 37, no. 2, pp. 75–85, 2018.
- J. Zhou, H. Huang, B. Liu, and X. Liu, “Normal estimation for 3d point clouds via local plane constraint and multi-scale selection,” Comput-Aided Des., vol. 129, p. 102916, 2020.
- Y. Ben-Shabat, M. Lindenbaum, and A. Fischer, “Nesti-net: Normal estimation for unstructured 3d point clouds using convolutional neural networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 10 112–10 120.
- J. Zhou, W. Jin, M. Wang, X. Liu, Z. Li, and Z. Liu, “Fast and accurate normal estimation for point clouds via patch stitching,” Comput.-Aided Des., vol. 142, p. 103121, 2022.
- Y. Ben-Shabat and S. Gould, “Deepfit: 3d surface fitting via neural network weighted least squares,” in Proceedings of the European Conference on Computer Vision (ECCV), 2020, pp. 20–34.
- R. Zhu, Y. Liu, Z. Dong, Y. Wang, T. Jiang, W. Wang, and B. Yang, “AdaFit: Rethinking learning-based normal estimation on point clouds,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021, pp. 6118–6127.
- J. Zhang, J.-J. Cao, H.-R. Zhu, D.-M. Yan, and X.-P. Liu, “Geometry guided deep surface normal estimation,” Comput.-Aided Des., vol. 142, p. 103119, 2022.
- J. Cao, H. Zhu, Y. Bai, J. Zhou, J. Pan, and Z. Su, “Latent tangent space representation for normal estimation,” IEEE Trans. Ind. Electron., vol. 69, no. 1, pp. 921–929, 2021.
- H. Zhou, H. Chen, Y. Zhang, M. Wei, H. Xie, J. Wang, T. Lu, J. Qin, and X.-P. Zhang, “Refine-Net: Normal refinement neural network for noisy point clouds,” IEEE Trans. Pattern Anal. Mach. Intell., 2022.
- K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.
- N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, “Indoor segmentation and support inference from rgbd images,” in Proceedings of the European Conference on Computer Vision (ECCV), 2012, pp. 746–760.
- F. Cazals and M. Pouget, “Estimating differential quantities using polynomial fitting of osculating jets,” Comput. Aided Geom. Des., vol. 22, no. 2, pp. 121–146, 2005.
- Q.-Y. Zhou, J. Park, and V. Koltun, “Fast global registration,” in Proceedings of the European Conference on Computer Vision (ECCV), 2016, pp. 766–782.
- R. Schnabel, R. Wahl, and R. Klein, “Efficient RANSAC for point-cloud shape detection,” Comput. Graph. Forum, vol. 26, no. 2, pp. 214–226, 2007.