Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
GPT-4o
Gemini 2.5 Pro Pro
o3 Pro
GPT-4.1 Pro
DeepSeek R1 via Azure Pro
2000 character limit reached

Geometric Prior Based Deep Human Point Cloud Geometry Compression (2305.01309v2)

Published 2 May 2023 in eess.IV and cs.CV

Abstract: The emergence of digital avatars has raised an exponential increase in the demand for human point clouds with realistic and intricate details. The compression of such data becomes challenging with overwhelming data amounts comprising millions of points. Herein, we leverage the human geometric prior in geometry redundancy removal of point clouds, greatly promoting the compression performance. More specifically, the prior provides topological constraints as geometry initialization, allowing adaptive adjustments with a compact parameter set that could be represented with only a few bits. Therefore, we can envisage high-resolution human point clouds as a combination of geometric priors and structural deviations. The priors could first be derived with an aligned point cloud, and subsequently the difference of features is compressed into a compact latent code. The proposed framework can operate in a play-and-plug fashion with existing learning based point cloud compression methods. Extensive experimental results show that our approach significantly improves the compression performance without deteriorating the quality, demonstrating its promise in a variety of applications.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. E. d’Eon, B. Harrison, T. Myers, and P. A. Chou, “8i voxelized full bodies (a voxelized point cloud dataset),” ISO/IEC JTC1/SC29/WG11, Geneva, Tech. Rep. M40059/M74006, January 2017.
  2. D. Graziosi, O. Nakagami, S. Kuma, A. Zaghetto, T. Suzuki, and A. Tabatabai, “An overview of ongoing point cloud compression standardization activities: Video-based (V-PCC) and geometry-based (G-PCC),” APSIPA Trans. Signal Inf. Process., vol. 9, p. e13, 2020.
  3. S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. César, P. A. Chou, R. A. Cohen, M. Krivokuca, S. Lasserre, Z. Li, J. Llach, K. Mammou, R. Mekuria, O. Nakagami, E. Siahaan, A. J. Tabatabai, A. M. Tourapis, and V. Zakharchenko, “Emerging mpeg standards for point cloud compression,” IEEE J. Emerg. Sel. Topics Circuits Syst., vol. 9, no. 1, pp. 133–148, 2019.
  4. MPEG 3D Graphics Coding, “V-PCC codec description,” ISO/IEC JTC 1/SC 29/WG 7, Tech. Rep. N00100, October 2020.
  5. MPEG 3D Graphics Coding, “G-PCC codec description,” ISO/IEC JTC 1/SC 29/WG 7, Tech. Rep. N0099, April 2021.
  6. G. J. Sullivan, J. Ohm, W. Han, and T. Wiegand, “Overview of the high efficiency video coding (HEVC) standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12, pp. 1649–1668, 2012.
  7. J. Ballé, V. Laparra, and E. P. Simoncelli, “End-to-end optimized image compression,” in 5th Int. Conf. Learn. Representations (ICLR), 2017.
  8. J. Ballé, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston, “Variational image compression with a scale hyperprior,” in 6th Int. Conf. Learn. Representations (ICLR), 2018.
  9. J. Wang, D. Ding, Z. Li, and Z. Ma, “Multiscale point cloud geometry compression,” in 31st Data Compression Conf. (DCC), 2021, pp. 73–82.
  10. M. Quach, G. Valenzise, and F. Dufaux, “Learning convolutional transforms for lossy point cloud geometry compression,” in IEEE Int. Conf. Image Process. (ICIP), 2019, pp. 4320–4324.
  11. J. Wang, D. Ding, Z. Li, X. Feng, C. Cao, and Z. Ma, “Sparse tensor-based multiscale representation for point cloud geometry compression,” IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–18, 2022.
  12. R. Mekuria, K. Blom, and P. Cesar, “Design, implementation, and evaluation of a point cloud codec for tele-immersive video,” IEEE Trans. Circuits Syst. Video Technol., vol. 27, no. 4, pp. 828–842, 2016.
  13. E. Ramalho, E. Peixoto, and E. Medeiros, “Silhouette 4d with context selection: Lossless geometry compression of dynamic point clouds,” IEEE Signal Process. Lett., vol. 28, pp. 1660–1664, 2021.
  14. E. Peixoto, “Intra-frame compression of point cloud geometry using dyadic decomposition,” IEEE Signal Process. Lett., vol. 27, pp. 246–250, 2020.
  15. X. Zhang, W. Gao, and S. Liu, “Implicit geometry partition for point cloud compression,” in Data Compression Conf. (DCC), 2020, pp. 73–82.
  16. C. Wang, W. Zhu, Y. Xu, Y. Xu, and L. Yang, “Point-voting based point cloud geometry compression,” in 23rd Int. Workshop Multimedia Signal Process. (MMSP), 2021, pp. 1–5.
  17. A. Ahmmed, M. Paul, M. M. Murshed, and D. Taubman, “Dynamic point cloud geometry compression using cuboid based commonality modeling framework,” in 2021 IEEE Int. Conf. Image Process. (ICIP), 2021, pp. 2159–2163.
  18. L. Li, Z. Li, S. Liu, and H. Li, “Efficient projected frame padding for video-based point cloud compression,” IEEE Trans. Multimedia, vol. 23, pp. 2806–2819, 2021.
  19. L. Li, Z. Li, V. Zakharchenko, J. Chen, and H. Li, “Advanced 3d motion prediction for video-based dynamic point cloud compression,” IEEE Trans. Image Process., vol. 29, pp. 289–302, 2020.
  20. J. Xiong, H. Gao, M. Wang, H. Li, K. N. Ngan, and W. Lin, “Efficient geometry surface coding in v-pcc,” IEEE Trans. Multimedia, pp. 1–1, 2022.
  21. P. de Oliveira Rente, C. Brites, J. Ascenso, and F. Pereira, “Graph-based static 3d point clouds geometry coding,” IEEE Trans. Multimedia, vol. 21, no. 2, pp. 284–299, 2019.
  22. W. Zhu, Y. Xu, D. Ding, Z. Ma, and M. Nilsson, “Lossy point cloud geometry compression via region-wise processing,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 12, pp. 4575–4589, 2021.
  23. W. Zhu, Z. Ma, Y. Xu, L. Li, and Z. Li, “View-dependent dynamic point cloud compression,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 2, pp. 765–781, 2021.
  24. M. Krivokuca, P. A. Chou, and M. Koroteev, “A volumetric approach to point cloud compression-part II: geometry compression,” IEEE Trans. Image Process., vol. 29, pp. 2217–2229, 2020.
  25. R. L. de Queiroz and P. A. Chou, “Motion-compensated compression of dynamic voxelized point clouds,” IEEE Trans. Image Process., vol. 26, no. 8, pp. 3886–3895, 2017.
  26. D. C. Garcia, T. A. da Fonseca, R. U. Ferreira, and R. L. de Queiroz, “Geometry coding for dynamic voxelized point clouds using octrees and multiple contexts,” IEEE Trans. Image Process., vol. 29, pp. 313–322, 2020.
  27. D. Thanou, P. A. Chou, and P. Frossard, “Graph-based compression of dynamic 3d point cloud sequences,” IEEE Trans. Image Process., vol. 25, no. 4, pp. 1765–1778, 2016.
  28. L. Huang, S. Wang, K. Wong, J. Liu, and R. Urtasun, “Octsqueeze: Octree-structured entropy model for lidar compression,” in IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR), 2020, pp. 1310–1320.
  29. Z. Que, G. Lu, and D. Xu, “Voxelcontext-net: An octree based framework for point cloud compression,” in IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR), 2021, pp. 6042–6051.
  30. C. Fu, G. Li, R. Song, W. Gao, and S. Liu, “Octattention: Octree-based large-scale contexts model for point cloud compression,” in 36th AAAI Conf. Artif. Intell. (AAAI), 2022, pp. 625–633.
  31. T. Fan, L. Gao, Y. Xu, D. Wang, and Z. Li, “Multiscale latent-guided entropy model for lidar point cloud compression,” arXiv:2209.12512, 2022.
  32. S. Biswas, J. Liu, K. Wong, S. Wang, and R. Urtasun, “Muscle: Multi sweep compression of lidar using deep entropy models,” in Conf. Neural Inf. Process. Syst. (NeurIPS), 2020.
  33. Z. Hu, G. Lu, and D. Xu, “FVC: A new framework towards deep video compression in feature space,” in IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR), 2021, pp. 1502–1511.
  34. G. Lu, W. Ouyang, D. Xu, X. Zhang, C. Cai, and Z. Gao, “DVC: an end-to-end deep video compression framework,” in IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR), 2019, pp. 11 006–11 015.
  35. M. Quach, G. Valenzise, and F. Dufaux, “Improved deep point cloud geometry compression,” in 22nd Int. Workshop Multimedia Signal Process. (MMSP), 2020, pp. 1–6.
  36. D. T. Nguyen, M. Quach, G. Valenzise, and P. Duhamel, “Learning-based lossless compression of 3d point cloud geometry,” in IEEE Int. Conf. Acoustics Speech Signal Process. (ICASSP), 2021, pp. 4220–4224.
  37. D. T. Nguyen, M. Quach, Giuseppe Valenzise, and P. Duhamel, “Multiscale deep context modeling for lossless point cloud geometry compression,” in IEEE Int. Conf. Multimedia Expo Workshops (ICMEW), 2021, pp. 1–6.
  38. A. F. R. Guarda, N. M. M. Rodrigues, and F. Pereira, “Deep learning-based point cloud geometry coding with resolution scalability,” in 22nd Int. Workshop Multimedia Signal Process. (MMSP), 2020, pp. 1–6.
  39. A. F. R. Guarda, N. M. M. Rodrigues, and F. Pereira, “Point cloud geometry scalable coding with a single end-to-end deep learning model,” in IEEE Int. Conf. Image Process. (ICIP), 2020, pp. 3354–3358.
  40. A. F. R. Guarda, N. M. M. Rodrigues, and F. Pereira, “Deep learning-based point cloud geometry coding: Rd control through implicit and explicit quantization,” in IEEE Int. Conf. Multimedia Expo (ICME) workshops, 2020, pp. 1–6.
  41. S. Milani, “ADAE: adversarial distributed source autoencoder for point cloud compression,” in IEEE Int. Conf. Image Process. (ICIP), 2021, pp. 3078–3082.
  42. J. Wang, H. Zhu, H. Liu, and Z. Ma, “Lossy point cloud geometry compression via end-to-end learning,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 12, pp. 4909–4923, 2021.
  43. A. Brock, T. Lim, J. M. Ritchie, and N. Weston, “Generative and discriminative voxel modeling with convolutional neural networks,” arXiv:1608.04236, 2016.
  44. R. Xue, J. Wang, and Z. Ma, “Efficient lidar point cloud geometry compression through neighborhood point attention,” arXiv:2208.12573, 2022.
  45. Z. Liang and F. Liang, “Transpcc: Towards deep point cloud compression via transformers,” in ICMR ’22: International Conference on Multimedia Retrieval, Newark, NJ, USA, June 27 - 30, 2022, 2022, pp. 1–5.
  46. Y. He, X. Ren, D. Tang, Y. Zhang, X. Xue, and Y. Fu, “Density-preserving deep point cloud compression,” in IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR), 2022, pp. 2323–2332.
  47. M. A. A. Muzaddid and W. J. Beksi, “Variable rate compression for raw 3d point clouds,” in Int. Conf. Robot. Autom. (ICRA), 2022, pp. 8748–8755.
  48. K. You and P. Gao, “Patch-based deep autoencoder for point cloud geometry compression,” in ACM Multimedia Asia, 2021, pp. 30:1–30:7.
  49. A. Akhtar, Z. Li, and G. V. der Auwera, “Inter-frame compression for dynamic point cloud geometry coding,” arXiv:2207.12554, 2022.
  50. T. Fan, L. Gao, Y. Xu, Z. Li, and D. Wang, “D-DPCC: deep dynamic point cloud compression via 3d motion prediction,” in 31st Int. Joint Conf. Artif. Intell. (IJCAI), 2022, pp. 898–904.
  51. W. Yang, Z. Chen, C. Chen, G. Chen, and K. K. Wong, “Deep face video inpainting via UV mapping,” IEEE Trans. Image Process., vol. 32, pp. 1145–1157, 2023.
  52. J. Lin, Y. Yuan, T. Shao, and K. Zhou, “Towards high-fidelity 3d face reconstruction from in-the-wild images using graph convolutional networks,” in IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR), 2020, pp. 5890–5899.
  53. F. Wimbauer, S. Wu, and C. Rupprecht, “De-rendering 3d objects in the wild,” in IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR), 2022, pp. 18 469–18 478.
  54. S. Wu, C. Rupprecht, and A. Vedaldi, “Unsupervised learning of probably symmetric deformable 3d objects from images in the wild,” in IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR), 2020, pp. 1–10.
  55. B. Chen, Z. Wang, B. Li, S. Wang, S. Wang, and Y. Ye, “Interactive face video coding: A generative compression framework,” arXiv:2302.09919, 2023.
  56. T. M. Hoang, J. Zhou, and Y. Fan, “Image compression with encoder-decoder matched semantic segmentation,” in IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR) workshops, 2020, pp. 160–161.
  57. P. Zhang, S. Wang, M. Wang, J. Li, X. Wang, and S. Kwong, “Rethinking semantic image compression: Scalable representation with cross-modality transfer,” IEEE Trans. Circuits Syst. Video Technol., vol. 33, no. 8, pp. 4441–4445, 2023.
  58. R. Hanocka, G. Metzer, R. Giryes, and D. Cohen-Or, “Point2mesh: a self-prior for deformable meshes,” ACM Trans. Graph., vol. 39, no. 4, p. 126, 2020.
  59. X. Wei, Z. Chen, Y. Fu, Z. Cui, and Y. Zhang, “Deep hybrid self-prior for full 3d mesh generation,” in IEEE/CVF Int. Conf. Computer Vision (ICCV), 2021, pp. 5785–5794.
  60. D. Smirnov, M. Bessmeltsev, and J. Solomon, “Learning manifold patch-based representations of man-made shapes,” in 9th Int. Conf. Learn. Representations (ICLR), 2021.
  61. M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black, “SMPL: a skinned multi-person linear model,” ACM Trans. Graph., vol. 34, no. 6, pp. 248:1–248:16, 2015.
  62. C. Xu, Y. Makihara, X. Li, and Y. Yagi, “Occlusion-aware human mesh model-based gait recognition,” IEEE Trans. Inf. Forensics Secur., vol. 18, pp. 1309–1321, 2023.
  63. X. Zuo, S. Wang, Q. Sun, M. Gong, and L. Cheng, “Self-supervised 3d human mesh recovery from noisy point clouds,” arXiv:2107.07539, 2021.
  64. B. Graham, M. Engelcke, and L. van der Maaten, “3d semantic segmentation with submanifold sparse convolutional networks,” in IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR), 2018, pp. 9224–9232.
  65. C. B. Choy, J. Gwak, and S. Savarese, “4d spatio-temporal convnets: Minkowski convolutional neural networks,” in IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR), 2019, pp. 3075–3084.
  66. J. Gwak, C. B. Choy, and S. Savarese, “Generative sparse detection networks for 3d single-shot object detection,” in 16th Europeon Conf. Computer Vision (ECCV), vol. 12349, 2020, pp. 297–313.
  67. D. Minnen, J. Ballé, and G. Toderici, “Joint autoregressive and hierarchical priors for learned image compression,” in Conf. Neural Inf. Process. Syst. (NeurIPS), 2018, pp. 10 794–10 803.
  68. Y. Xu, Y. Lu, and Z. Wen, “Owlii dynamic human mesh sequence dataset,” ISO/IEC JTC1/SC29 WG11, Macau, Tech. Rep. M41658, October 2017.
  69. T. Yu, Z. Zheng, K. Guo, P. Liu, Q. Dai, and Y. Liu, “Function4d: Real-time human volumetric capture from very sparse consumer RGBD sensors,” in IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR), 2021, pp. 5746–5756.
  70. M. Krivokuća, P. A. Chou, and P. Savill, “8i voxelized surface light field (8iVSLF) dataset,” ISO/IEC JTC1/SC29 WG11, Ljubljana, Tech. Rep. M42914, July 2018.
  71. 3DG, “Common test conditions for V3C and V-PCC,” ISO/IEC JTC 1/SC 29/WG 11, Tech. Rep. N19518, July 2020.
  72. 3DG, “Common test conditions for G-PCC,” ISO/IEC JTC 1/SC 29/WG 11, Tech. Rep. N19584, July 2020.
  73. S. Zuffi, A. Kanazawa, D. W. Jacobs, and M. J. Black, “3d menagerie: Modeling the 3d shape and pose of animals,” in IEEE Int. Conf. Comput. Vision Pattern Recognit. (CVPR), 2017, pp. 6365–6373.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now