Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

3D Shape Knowledge Graph for Cross-domain 3D Shape Retrieval (2210.15136v2)

Published 27 Oct 2022 in cs.CV

Abstract: The surge in 3D modeling has led to a pronounced research emphasis on the field of 3D shape retrieval. Numerous contemporary approaches have been put forth to tackle this intricate challenge. Nevertheless, effectively addressing the intricacies of cross-modal 3D shape retrieval remains a formidable undertaking, owing to inherent modality-based disparities. This study presents an innovative notion, termed "geometric words", which functions as elemental constituents for representing entities through combinations. To establish the knowledge graph, we employ geometric words as nodes, connecting them via shape categories and geometry attributes. Subsequently, we devise a unique graph embedding method for knowledge acquisition. Finally, an effective similarity measure is introduced for retrieval purposes. Importantly, each 3D or 2D entity can anchor its geometric terms within the knowledge graph, thereby serving as a link between cross-domain data. As a result, our approach facilitates multiple cross-domain 3D shape retrieval tasks. We evaluate the proposed method's performance on the ModelNet40 and ShapeNetCore55 datasets, encompassing scenarios related to 3D shape retrieval and cross-domain retrieval. Furthermore, we employ the established cross-modal dataset (MI3DOR) to assess cross-modal 3D shape retrieval. The resulting experimental outcomes, in conjunction with comparisons against state-of-the-art techniques, clearly highlight the superiority of our approach.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (81)
  1. Deep learning for 3d point clouds: A survey. IEEE transactions on pattern analysis and machine intelligence. 2020;43(12):4338-64.
  2. Pixel2Mesh: 3D mesh model generation via image guided deformation. IEEE transactions on pattern analysis and machine intelligence. 2020;43(10):3600-13.
  3. Multi-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE international conference on computer vision; 2015. p. 945-53.
  4. Deep point-to-subspace metric learning for sketch-based 3D shape retrieval. Pattern Recognit. 2019;96.
  5. Goldfeder C, Allen PK. Autotagging to improve text search for 3d models. In: ACM/IEEE Joint Conference on Digital Libraries, JCDL 2008, Pittsburgh, PA, USA, June 16-20, 2008; 2008. p. 355-8.
  6. Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2017. p. 652-60.
  7. MLVCNN: Multi-loop-view convolutional neural network for 3D shape retrieval. In: Proceedings of the AAAI conference on artificial intelligence. vol. 33; 2019. p. 8513-20.
  8. Pvnet: A joint convolutional network of point cloud and multi-view for 3d shape recognition. In: 2018 ACM Multimedia Conference on Multimedia Conference. ACM; 2018. p. 1310-8.
  9. Deep Correlated Holistic Metric Learning for Sketch-Based 3D Shape Retrieval. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society. 2018;27(7):3374.
  10. Transfer joint matching for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2014. p. 1410-7.
  11. Joint geometrical and statistical alignment for visual domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. p. 1859-67.
  12. 3d shapenets: A deep representation for volumetric shapes. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2015. p. 1912-20.
  13. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In: Advances in neural information processing systems; 2017. p. 5099-108.
  14. Garro V, Giachetti A. Scale Space Graph Representation and Kernel Matching for Non Rigid and Textured 3D Shape Retrieval. IEEE Trans Pattern Anal Mach Intell. 2016;38(6):1258-71.
  15. DeepShape: Deep-Learned Shape Descriptor for 3D Shape Retrieval. IEEE Trans Pattern Anal Mach Intell. 2017;39(7):1335-45.
  16. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (tog). 2019;38(5):1-12.
  17. On Visual Similarity Based 3D Model Retrieval. Computer Graphics Forum. 2010;22(3):223-32.
  18. GIFT: Towards Scalable 3D Shape Retrieval. IEEE Transactions on Multimedia. 2017 June;19(6):1257-71.
  19. Ensemble of panorama-based convolutional neural networks for 3d model classification and retrieval. Computers & Graphics. 2018;71:208-18.
  20. Local Spectral Graph Convolution for Point Set Feature Learning. In: Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part IV. vol. 11208 of Lecture Notes in Computer Science. Springer; 2018. p. 56-71.
  21. Rgcnn: Regularized graph cnn for point cloud segmentation. In: Proceedings of the 26th ACM international conference on Multimedia; 2018. p. 746-54.
  22. Shi W, Rajkumar R. Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. IEEE; 2020. p. 1708-16.
  23. Exploiting edge-oriented reasoning for 3d point-based scene graph analysis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2021. p. 9705-15.
  24. Tn-zstad: Transferable network for zero-shot temporal activity detection. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022;45(3):3848-61.
  25. When object detection meets knowledge distillation: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023.
  26. Video pivoting unsupervised multi-modal machine translation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022;45(3):3918-32.
  27. Category alignment adversarial learning for cross-modal retrieval. IEEE Transactions on Knowledge and Data Engineering. 2022;35(5):4527-38.
  28. Deep collaborative embedding for social image understanding. IEEE transactions on pattern analysis and machine intelligence. 2018;41(9):2070-83.
  29. Li Z, Tang J. Weakly supervised deep metric learning for community-contributed image retrieval. IEEE Transactions on Multimedia. 2015;17(11):1989-99.
  30. Image-based 3D model retrieval using manifold learning. Journal of Zhejiang University Science C. 2018;19(11):1397-408.
  31. Joint embeddings of shapes and images via CNN image purification. international conference on computer graphics and interactive techniques. 2015;34(6):234.
  32. SeqViews2SeqLabels: Learning 3D Global Features via Aggregating Sequential Views by RNN With Attention. IEEE Transactions on Image Processing. 2019;28(2):658-72.
  33. 3D Model Retrieval Based on a 3D Shape Knowledge Graph. IEEE Access. 2020;8:142632-41.
  34. Physical primitive decomposition. In: Proceedings of the European Conference on Computer Vision (ECCV); 2018. p. 3-19.
  35. PointDCCNet: 3D Object Categorization Network Using Point Cloud Decomposition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2021. p. 2200-8.
  36. RGB-D images for object segmentation, localization and recognition in indoor scenes using feature descriptor and Hough voting. In: 2020 17th International Bhurban Conference on Applied Sciences and Technology (IBCAST). IEEE; 2020. p. 290-5.
  37. Semantic part segmentation method based 3D object pose estimation with RGB-D images for bin-picking. Robotics and Computer-Integrated Manufacturing. 2021;68:102086.
  38. An Unequal Learning Approach for 3D Point Cloud Segmentation. IEEE Transactions on Industrial Informatics. 2020.
  39. GVCNN: Group-view convolutional neural networks for 3D shape recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2018. p. 264-72.
  40. Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2019. p. 909-18.
  41. Learning hierarchical shape segmentation and labeling from online repositories. arXiv preprint arXiv:170501661. 2017.
  42. Partnet: A recursive part decomposition network for fine-grained and hierarchical shape segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2019. p. 9491-500.
  43. 3D shape segmentation with projective convolutional networks. In: proceedings of the IEEE conference on computer vision and pattern recognition; 2017. p. 3779-88.
  44. CTNet: Context-based tandem network for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2021;44(12):9904-17.
  45. Monet: Unsupervised scene decomposition and representation. arXiv preprint arXiv:190111390. 2019.
  46. Hologan: Unsupervised learning of 3d representations from natural images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision; 2019. p. 7588-97.
  47. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham: Springer International Publishing; 2015. p. 234-41.
  48. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Transactions on Pattern Analysis Machine Intelligence. 2017:1-1.
  49. Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. p. 2881-90.
  50. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:170605587. 2017.
  51. Principal component analysis. Chemometrics and intelligent laboratory systems. 1987;2(1-3):37-52.
  52. Efficient 3D shape matching and retrieval using a concrete radialized spherical projection representation. Pattern Recognition. 2007;40(9):2437 2452.
  53. Salient object detection with pyramid attention and salient edges. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2019. p. 1448-57.
  54. Identity mappings in deep residual networks. In: European conference on computer vision. Springer; 2016. p. 630-45.
  55. Inductive representation learning on large graphs. In: Advances in neural information processing systems; 2017. p. 1024-34.
  56. 3D model retrieval using weighted bipartite graph matching. Signal Processing: Image Communication. 2011;26(1):39-47.
  57. Monocular Image Based 3D Model Retrieval. In: 12th Eurographics Workshop on 3D Object Retrieval, 3DORs. Eurographics Association; 2019. p. 103-10.
  58. View-based 3-D model retrieval: a benchmark. IEEE transactions on cybernetics. 2017;48(3):916-28.
  59. Rotation invariant spherical harmonic representation of 3 d shape descriptors. In: Symposium on geometry processing. vol. 6; 2003. p. 156-64.
  60. Volumetric and multi-view cnns for object classification on 3d data. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. p. 5648-56.
  61. Dynamic graph cnn for learning on point clouds. arXiv preprint arXiv:180107829. 2018.
  62. On visual similarity based 3D model retrieval. In: Computer graphics forum. vol. 22. Wiley Online Library; 2003. p. 223-32.
  63. Voxnet: An interactive, rapidly-deployable acoustic monitoring platform. In: 2008 International Conference on Information Processing in Sensor Networks (ipsn 2008). IEEE; 2008. p. 371-82.
  64. Generative and Discriminative Voxel Modeling with Convolutional Neural Networks. Computer Science. 2016.
  65. Latent-MVCNN: 3D shape recognition using multiple views from pre-defined or random viewpoints. Neural Processing Letters. 2020;52:581-602.
  66. Boosting multi-view convolutional neural networks for 3d object recognition via view saliency. In: Chinese Conference on Image and Graphics Technologies. Springer; 2017. p. 199-209.
  67. View n-gram network for 3D object retrieval. In: Proceedings of the IEEE International Conference on Computer Vision; 2019. p. 7515-24.
  68. Pct: Point cloud transformer. Computational Visual Media. 2021;7:187-99.
  69. Multi-level view associative convolution network for view-based 3D model retrieval. IEEE Transactions on Circuits and Systems for Video Technology. 2021;32(4):2264-78.
  70. Ganin Y, Lempitsky VS. Unsupervised Domain Adaptation by Backpropagation. In: Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015. vol. 37 of JMLR Workshop and Conference Proceedings. JMLR.org; 2015. p. 1180-9.
  71. Deep Transfer Learning with Joint Adaptation Networks. In: Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017. vol. 70 of Proceedings of Machine Learning Research. PMLR; 2017. p. 2208-17.
  72. Joint Geometrical and Statistical Alignment for Visual Domain Adaptation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. IEEE Computer Society; 2017. p. 5150-8.
  73. Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE international conference on computer vision; 2013. p. 2200-7.
  74. Self-supervised image-based 3d model retrieval. ACM Transactions on Multimedia Computing, Communications and Applications. 2023;19(2):1-18.
  75. Instance-prototype similarity consistency for unsupervised 2D image-based 3D model retrieval. Information Processing & Management. 2023;60(4):103372.
  76. Visual domain adaptation with manifold embedded distribution alignment. In: Proceedings of the 26th ACM international conference on Multimedia; 2018. p. 402-10.
  77. Pointcnn: Convolution on x-transformed points. Advances in neural information processing systems. 2018;31.
  78. Rotationnet: Joint object categorization and pose estimation using multiviews from unsupervised viewpoints. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2018. p. 5010-9.
  79. Deep Residual Learning for Image Recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society; 2016. p. 770-8.
  80. Peng Y, Chi J. Unsupervised Cross-Media Retrieval Using Domain Adaptation With Scene Graph. IEEE Trans Circuits Syst Video Technol. 2020;30(11):4368-79.
  81. Text2shape: Generating shapes from natural language by learning joint embeddings. In: Asian Conference on Computer Vision. Springer; 2018. p. 100-16.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now