Hyperbolic Hierarchical Knowledge Graph Embeddings for Link Prediction in Low Dimensions (2204.13704v2)
Abstract: Knowledge graph embeddings (KGE) have been validated as powerful methods for inferring missing links in knowledge graphs (KGs) that they typically map entities into Euclidean space and treat relations as transformations of entities. Recently, some Euclidean KGE methods have been enhanced to model semantic hierarchies commonly found in KGs, improving the performance of link prediction. To embed hierarchical data, hyperbolic space has emerged as a promising alternative to traditional Euclidean space, offering high fidelity and lower memory consumption. Unlike Euclidean, hyperbolic space provides countless curvatures to choose from. However, it is difficult for existing hyperbolic KGE methods to obtain the optimal curvature settings manually, thereby limiting their ability to effectively model semantic hierarchies. To address this limitation, we propose a novel KGE model called $\textbf{Hyp}$erbolic $\textbf{H}$ierarchical $\textbf{KGE}$ (HypHKGE). This model introduces attention-based learnable curvatures for hyperbolic space, which helps preserve rich semantic hierarchies. Furthermore, to utilize the preserved hierarchies for inferring missing links, we define hyperbolic hierarchical transformations based on the theory of hyperbolic geometry, including both inter-level and intra-level modeling. Experiments demonstrate the effectiveness of the proposed HypHKGE model on the three benchmark datasets (WN18RR, FB15K-237, and YAGO3-10). The source code will be publicly released at https://github.com/wjzheng96/HypHKGE.
- James W Anderson. 2006. Hyperbolic geometry. Springer Science & Business Media.
- Ben Andrews and Christopher Hopper. 2010. The Ricci flow in Riemannian geometry: a complete proof of the differentiable 1/4-pinching sphere theorem. springer.
- Multi-relational poincaré graph embeddings. In NeurIPS. 4463–4473.
- Freebase: a collaboratively created graph database for structuring human knowledge. In ACM SIGMOD. 1247–1250.
- William M Boothby. 1986. An introduction to differentiable manifolds and Riemannian geometry. Academic press.
- Translating embeddings for modeling multi-relational data. In NeurIPS. 1–9.
- Toward an architecture for never-ending language learning. In AAAI.
- Low dimensional hyperbolic knowledge graph embeddings. In ACL. 6901–6914.
- Hyperbolic graph convolutional neural networks. In NeurIPS. 4869–4880.
- Convolutional 2d knowledge graph embeddings. In AAAI.
- Hyperbolic neural networks. In NeurIPS. 5345–5355.
- Hyperbolic knowledge graph embeddings for knowledge base completion. In ESWC. 199–214.
- Hyperbolic geometry of complex networks. Physical Review E 82, 3 (2010), 036106.
- John M Lee. 2013. Smooth manifolds. In Introduction to Smooth Manifolds. 1–31.
- Learning entity and relation embeddings for knowledge graph completion. In AAAI.
- DensE: An enhanced non-commutative representation for knowledge graph embedding with adaptive semantic hierarchy. Neurocomputing 476 (2022), 115–125.
- Yago3: A knowledge base from multilingual wikipedias. In CIDR.
- 5* knowledge graph embeddings with projective transformations. In AAAI. 9064–9072.
- Maximillian Nickel and Douwe Kiela. 2017. Poincaré embeddings for learning hierarchical representations. In NeurIPS. 6338–6347.
- Zhe Pan and Peng Wang. 2021. Hyperbolic hierarchy-aware knowledge graph embedding for link prediction. In EMNLP-Findings 2021. 2941–2948.
- Rishi Sonthalia and Anna Gilbert. 2020. Tree! I am no Tree! I am a low dimensional Hyperbolic Embedding. In NeurIPS. 845–856.
- Michael Spivak. 1975. A comprehensive introduction to differential geometry. Vol. 5. Publish or Perish, Incorporated.
- Rotate: Knowledge graph embedding by relational rotation in complex space. In ICLR. 1–18.
- Orthogonal relation transforms with graph context modeling for knowledge graph embedding. In ACL. 2713–2722.
- Poincaré Glove: Hyperbolic word embeddings. In ICLR.
- Kristina Toutanova. 2015. Observed versus latent features for knowledge base and text inference. In ACL-IJCNLP. 57–66.
- Complex embeddings for simple link prediction. In ICML. 2071–2080.
- Mixed-Curvature Multi-Relational Graph Neural Network for Knowledge Graph Completion. In WWW. 1761–1771.
- Knowledge graph embedding by translating on hyperplanes. In AAAI.
- TransG: A generative mixture model for knowledge graph embedding. In ACL. 2316–2325.
- Representation Learning of Knowledge Graphs with Hierarchical Types.. In IJCAI. 2965–2971.
- Canran Xu and Ruijiang Li. 2019. Relation embedding with dihedral group in knowledge graph. In ACL. 263–272.
- Learning hierarchy-aware knowledge graph embeddings for link prediction. In AAAI. 3065–3072.
- Knowledge graph embedding with hierarchical relation structure. In EMNLP. 3198–3207.
- Wenjie Zheng (9 papers)
- Wenxue Wang (1 paper)
- Shu Zhao (31 papers)
- Fulan Qian (1 paper)