RESSCAL3D: Resolution Scalable 3D Semantic Segmentation of Point Clouds (2404.06863v1)
Abstract: While deep learning-based methods have demonstrated outstanding results in numerous domains, some important functionalities are missing. Resolution scalability is one of them. In this work, we introduce a novel architecture, dubbed RESSCAL3D, providing resolution-scalable 3D semantic segmentation of point clouds. In contrast to existing works, the proposed method does not require the whole point cloud to be available to start inference. Once a low-resolution version of the input point cloud is available, first semantic predictions can be generated in an extremely fast manner. This enables early decision-making in subsequent processing steps. As additional points become available, these are processed in parallel. To improve performance, features from previously computed scales are employed as prior knowledge at the current scale. Our experiments show that RESSCAL3D is 31-62% faster than the non-scalable baseline while keeping a limited impact on performance. To the best of our knowledge, the proposed method is the first to propose a resolution-scalable approach for 3D semantic segmentation of point clouds based on deep learning.
- “Multi-scale end-to-end learning for point cloud geometry compression,” in 2022 IEEE International Conference on Image Processing (ICIP). IEEE, 2022, pp. 2107–2111.
- “Deep-learning-based lossless image coding,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 7, pp. 1829–1842, 2019.
- “Gsnet: Joint vehicle pose and shape reconstruction with geometrical and scene-aware supervision,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XV 16. Springer, 2020, pp. 515–532.
- “Mono6d: Monocular vehicle 6d pose estimation with 3d priors,” in 2022 IEEE International Conference on Image Processing (ICIP). IEEE, 2022, pp. 2187–2191.
- “Point transformer,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 16259–16268.
- “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” Advances in neural information processing systems, vol. 30, 2017.
- “Efficientdet: Scalable and efficient object detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10781–10790.
- “Runtime neural pruning,” Advances in neural information processing systems, vol. 30, 2017.
- “Masklayer: Enabling scalable deep learning solutions by training embedded feature sets,” Neural Networks, vol. 137, pp. 43–53, 2021.
- “Scalable intraband and composite wavelet-based coding of semiregular meshes,” IEEE Transactions on Multimedia, vol. 12, no. 8, pp. 773–789, 2010.
- “Hierarchical oriented predictions for resolution scalable lossless and near-lossless compression of ct and mri biomedical images,” IEEE Transactions on image processing, vol. 21, no. 5, pp. 2641–2652, 2012.
- “Scalable wavelet-based coding of irregular meshes with interactive region-of-interest support,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 29, no. 7, pp. 2067–2081, 2018.
- “Deep learning-based point cloud geometry coding with resolution scalability,” in 2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP). IEEE, 2020, pp. 1–6.
- “Low power, low latency 3d perception for xr,” presented at SPIE AR—VR—MR, 2023.
- “Voxelsensors,” https://voxelsensors.com/, Accessed: 2023-02-17.
- “Pointcnn: Convolution on x-transformed points,” Advances in neural information processing systems, vol. 31, 2018.
- “Kpconv: Flexible and deformable convolution for point clouds,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 6411–6420.
- “Pointweb: Enhancing local neighborhood features for point cloud processing,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 5565–5573.
- “Pointnext: Revisiting pointnet++ with improved training and scaling strategies,” arXiv preprint arXiv:2206.04670, 2022.
- “3d semantic parsing of large-scale indoor spaces,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1534–1543.