Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

HyCubE: Efficient Knowledge Hypergraph 3D Circular Convolutional Embedding (2402.08961v3)

Published 14 Feb 2024 in cs.AI

Abstract: Knowledge hypergraph embedding models are usually computationally expensive due to the inherent complex semantic information. However, existing works mainly focus on improving the effectiveness of knowledge hypergraph embedding, making the model architecture more complex and redundant. It is desirable and challenging for knowledge hypergraph embedding to reach a trade-off between model effectiveness and efficiency. In this paper, we propose an end-to-end efficient knowledge hypergraph embedding model, HyCubE, which designs a novel 3D circular convolutional neural network and the alternate mask stack strategy to enhance the interaction and extraction of feature information comprehensively. Furthermore, our proposed model achieves a better trade-off between effectiveness and efficiency by adaptively adjusting the 3D circular convolutional layer structure to handle n-ary knowledge tuples of different arities with fewer parameters. In addition, we use a knowledge hypergraph 1-N multilinear scoring way to accelerate the model training efficiency further. Finally, extensive experimental results on all datasets demonstrate that our proposed model consistently outperforms state-of-the-art baselines, with an average improvement of 8.22% and a maximum improvement of 33.82% across all metrics. Meanwhile, HyCubE is 6.12x faster, GPU memory usage is 52.67% lower, and the number of parameters is reduced by 85.21% compared with the average metric of the latest state-of-the-art baselines.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Self-attention presents low-dimensional knowledge graph embeddings for link prediction. Knowledge-Based Systems 260 (2023), 110–124.
  2. Knowledge Graph Reasoning over Entities and Numerical Values. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’23). 57–68.
  3. TuckER: Tensor Factorization for Knowledge Graph Completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (EMNLP’19). 5185–5194.
  4. Translating embeddings for modeling multi-relational data. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NeurIPS’13). 2787–2795.
  5. Efficient Join Order Selection Learning with Graph-based Representation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’22). 97–107.
  6. HittER: Hierarchical Transformers for Knowledge Graph Embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP’21). 10395–10407.
  7. Explainable Link Prediction in Knowledge Hypergraphs. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management (CIKM’22). 262–271.
  8. PosKHG: A Position-Aware Knowledge Hypergraph Model for Link Prediction. Data Science and Engineering 8, 2 (2023), 135–145.
  9. Convolutional 2D Knowledge Graph Embeddings. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI’18). 1811–1818.
  10. Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research 12, 7 (2011).
  11. Knowledge hypergraphs: prediction beyond binary relations. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence (IJCAI’21). 2191–2197.
  12. Knowledge hypergraph embedding meets relational algebra. Journal of Machine Learning Research 24, 105 (2023), 1–34.
  13. Message Passing for Hyper-Relational Knowledge Graphs. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP’20). 7346–7359.
  14. NeuInfer: Knowledge inference on n-ary facts. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL’20). 6141–6151.
  15. Link prediction on n-ary relational data based on relatedness evaluation. IEEE Transactions on Knowledge and Data Engineering 35, 1 (2023), 672–685.
  16. Link prediction on n-ary relational data. In Proceedings of the 2019 World Wide Web Conference (WWW’19). 583–593.
  17. Line: Logical query reasoning over hierarchical knowledge graphs. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’22). 615–625.
  18. Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning (ICML’15). 448–456.
  19. Seyed Mehran Kazemi and David Poole. 2018. SimplE embedding for link prediction in knowledge graphs. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NeurIPS’18). 4289–4300.
  20. Exploiting Relation-aware Attribute Representation Learning in Knowledge Graph Embedding for Numerical Reasoning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’23). 1086–1096.
  21. Brenden M Lake and Marco Baroni. 2023. Human-like systematic generalization through a meta-learning neural network. Nature 623, 7985 (2023), 115–121.
  22. Zhao Li. 2023. Knowledge Hypergraph Reasoning Based on Representation Learning. In International Conference on Database Systems for Advanced Applications (DASFAA’23). 743–747.
  23. Mask and reason: Pre-training knowledge graph transformers for complex logical queries. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’22). 1120–1130.
  24. Generalizing tensor decomposition for n-ary relational knowledge bases. In Proceedings of The Web Conference 2020 (WWW’20). 1104–1114.
  25. Role-aware modeling for n-Ary relational knowledge bases. In Proceedings of The Web Conference 2021 (WWW’21). 2660–2671.
  26. Criteria Tell You More than Ratings: Criteria Preference-Aware Light Graph Convolution for Effective Multi-Criteria Recommendation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’23). 1808–1819.
  27. Knowledge Graph Embedding with Atrous Convolution and Residual Learning. In Proceedings of the 28th International Conference on Computational Linguistics (COLING’20). 1532–1543.
  28. Beyond triplets: hyper-relational knowledge graph embedding for link prediction. In Proceedings of The Web Conference 2020 (WWW’20). 1885–1896.
  29. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1 (2014), 1929–1958.
  30. Complex embeddings for simple link prediction. In Proceedings of the 33rd International Conference on International Conference on Machine Learning (ICML’16). 2071–2080.
  31. Interacte: Improving convolution-based knowledge graph embeddings by increasing feature interactions. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI’20). 3009–3016.
  32. HyConvE: A Novel Embedding Model for Knowledge Hypergraph Link Prediction with Convolutional Neural Networks. In Proceedings of the ACM Web Conference 2023 (WWW’23). 188–198.
  33. Omnidirectional CNN for Visual Place Recognition and Navigation. In IEEE International Conference on Robotics and Automation (ICRA’18). 2341–2348.
  34. Knowledge Graph Embedding by Translating on Hyperplanes. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI’14). 1112–1119.
  35. On the representation and embedding of knowledge bases beyond binary relations. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI’16). 1300–1307.
  36. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In Proceedings of the third International Conference on Learning Representations (ICLR’15). 809–816.
  37. Scalable instance reconstruction in knowledge bases via relatedness affiliated embedding. In Proceedings of the 2018 World Wide Web Conference (WWW’18). 1185–1194.
  38. Dynamic relation learning for link prediction in knowledge hypergraphs. Applied Intelligence 53, 22 (2023), 26580–26591.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets