Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Multi-level Shared Knowledge Guided Learning for Knowledge Graph Completion (2405.06696v1)

Published 8 May 2024 in cs.CL and cs.AI

Abstract: In the task of Knowledge Graph Completion (KGC), the existing datasets and their inherent subtasks carry a wealth of shared knowledge that can be utilized to enhance the representation of knowledge triplets and overall performance. However, no current studies specifically address the shared knowledge within KGC. To bridge this gap, we introduce a multi-level Shared Knowledge Guided learning method (SKG) that operates at both the dataset and task levels. On the dataset level, SKG-KGC broadens the original dataset by identifying shared features within entity sets via text summarization. On the task level, for the three typical KGC subtasks - head entity prediction, relation prediction, and tail entity prediction - we present an innovative multi-task learning architecture with dynamically adjusted loss weights. This approach allows the model to focus on more challenging and underperforming tasks, effectively mitigating the imbalance of knowledge sharing among subtasks. Experimental results demonstrate that SKG-KGC outperforms existing text-based methods significantly on three well-known datasets, with the most notable improvement on WN18RR.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. TuckER: Tensor factorization for knowledge graph completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5185–5194, Hong Kong, China. Association for Computational Linguistics.
  2. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, SIGMOD ’08, pages 1247–1250, Vancouver, Canada. Association for Computing Machinery.
  3. Translating embeddings for modeling multi-relational data. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2, NIPS’13, pages 2787–2795. Curran Associates Inc.
  4. Building a morpho-semantic knowledge graph for arabic information retrieval. Information Processing & Management, 57(6):102124.
  5. Dipping PLMs sauce: Bridging structure and text for effective knowledge graph completion via conditional soft prompting. In Findings of the Association for Computational Linguistics: ACL 2023, pages 11489–11503, Toronto, Canada. Association for Computational Linguistics.
  6. Inductive entity representations from text via link prediction. In Proceedings of the Web Conference 2021, WWW ’21, pages 798–808, Ljubljana, Slovenia. Association for Computing Machinery.
  7. Convolutional 2D knowledge graph embeddings. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, volume 32 of AAAI’18/IAAI’18/EAAI’18, pages 1811–1818, New Orleans, Louisiana, USA. AAAI Press.
  8. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  9. A novel embedding model for knowledge graph completion based on multi-task learning. In Database Systems for Advanced Applications: 26th International Conference, DASFAA 2021, pages 240–255, Taipei, Taiwan. Springer-Verlag.
  10. Enhanced multi-task learning and knowledge graph-based recommender system. IEEE Transactions on Knowledge and Data Engineering, 35(10):10281–10294.
  11. Link prediction with attention applied on multiple knowledge graph embedding models. In Proceedings of the ACM Web Conference 2023, WWW ’23, pages 2600–2610, New York, NY, USA. Association for Computing Machinery.
  12. Type-sensitive knowledge base inference without explicit type supervision. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 75–80, Melbourne, Australia. Association for Computational Linguistics.
  13. Knowledge base completion: Baseline strikes back (again). CoRR, abs/2005.00804v3.
  14. Unifying structure and language semantic for efficient contrastive knowledge graph completion with structured entity anchors. arXiv preprint arXiv:2311.04250.
  15. Multi-task learning for knowledge graph completion with pre-trained language models. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1737–1743, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  16. ALBERT: A lite BERT for self-supervised learning of language representations. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia. OpenReview.net.
  17. Multi-task pre-training language model for semantic network completion. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(11).
  18. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2):318–327.
  19. Do pre-trained models benefit knowledge graph completion? A reliable evaluation and a reasonable approach. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3570–3581, Dublin, Ireland. Association for Computational Linguistics.
  20. Rada Mihalcea and Paul Tarau. 2004. TextRank: Bringing order into text. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 404–411, Barcelona, Spain. Association for Computational Linguistics.
  21. George A. Miller. 1995. WordNet: A lexical database for english. Communications of the ACM, 38(11):39–41.
  22. Electrocardiogram heartbeat classification based on a deep convolutional neural network and focal loss. Computers in Biology and Medicine, 123:103866.
  23. Sequence-to-sequence knowledge graph completion and question answering. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2814–2828, Dublin, Ireland. Association for Computational Linguistics.
  24. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4498–4507, Online. Association for Computational Linguistics.
  25. RotatE: Knowledge graph embedding by relational rotation in complex space. In 7th International Conference on Learning Representations, New Orleans, LA, USA. OpenReview.net.
  26. Multi-task learning and improved TextRank for knowledge graph completion. Entropy, 24(10):1495.
  27. Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, pages 57–66, Beijing, China. Association for Computational Linguistics.
  28. Complex embeddings for simple link prediction. In Proceedings of the 33rd International Conference on International Conference on Machine Learning, volume 48 of ICML’16, pages 2071–2080, New York, NY, USA. Proceedings of Machine Learning Research.
  29. Structure-augmented text representation learning for efficient knowledge graph completion. In Proceedings of the Web Conference 2021, WWW ’21, pages 1737–1748, Ljubljana, Slovenia. Association for Computing Machinery.
  30. SimKGC: Simple contrastive knowledge graph completion with pre-trained language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4281–4294, Dublin, Ireland. Association for Computational Linguistics.
  31. KEPLER: A unified model for knowledge embedding and pre-trained language representation. Transactions of the Association for Computational Linguistics, 9:176–194.
  32. Representation learning of knowledge graphs with entity descriptions. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, pages 2659–2665, Phoenix, Arizona. AAAI Press.
  33. From discrimination to generation: Knowledge graph completion with generative transformer. In Companion Proceedings of the Web Conference 2022, WWW ’22, pages 162–165, Virtual Event, Lyon, France. Association for Computing Machinery.
  34. Embedding entities and relations for learning and inference in knowledge bases. In 3rd International Conference on Learning Representations, San Diego, CA, USA.
  35. Improving multilingual sentence embedding using bi-directional dual encoder with additive margin softmax. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 5370–5378, Macao, China. International Joint Conferences on Artificial Intelligence Organization.
  36. KG-BERT: BERT for knowledge graph completion. CoRR, abs/1909.03193v2.
  37. Exploring large language models for knowledge graph completion. CoRR, abs/2308.13916v4.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now