Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Two Heads Are Better Than One: Integrating Knowledge from Knowledge Graphs and Large Language Models for Entity Alignment (2401.16960v1)

Published 30 Jan 2024 in cs.CL and cs.AI

Abstract: Entity alignment, which is a prerequisite for creating a more comprehensive Knowledge Graph (KG), involves pinpointing equivalent entities across disparate KGs. Contemporary methods for entity alignment have predominantly utilized knowledge embedding models to procure entity embeddings that encapsulate various similarities-structural, relational, and attributive. These embeddings are then integrated through attention-based information fusion mechanisms. Despite this progress, effectively harnessing multifaceted information remains challenging due to inherent heterogeneity. Moreover, while LLMs have exhibited exceptional performance across diverse downstream tasks by implicitly capturing entity semantics, this implicit knowledge has yet to be exploited for entity alignment. In this study, we propose a LLM-enhanced Entity Alignment framework (LLMEA), integrating structural knowledge from KGs with semantic knowledge from LLMs to enhance entity alignment. Specifically, LLMEA identifies candidate alignments for a given entity by considering both embedding similarities between entities across KGs and edit distances to a virtual equivalent entity. It then engages an LLM iteratively, posing multiple multi-choice questions to draw upon the LLM's inference capability. The final prediction of the equivalent entity is derived from the LLM's output. Experiments conducted on three public datasets reveal that LLMEA surpasses leading baseline models. Additional ablation studies underscore the efficacy of our proposed framework.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. Translating Embeddings for Modeling Multi-Relational Data. In Proceedings of the 26th International Conference on Neural Information Processing Systems. 2787–2795.
  2. Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 1860–1874.
  3. JarKA: Modeling Attribute Interactions for Cross-lingual Knowledge Alignment. In Advances in Knowledge Discovery and Data Mining. 845–856.
  4. Multilingual Knowledge Graph Embeddings for Cross-Lingual Knowledge Alignment. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 1511–1517.
  5. Deep Reinforcement Learning from Human Preferences. In Advances in Neural Information Processing Systems.
  6. Knowledge Graph Enhanced Contextualized Attention-Based Network for Responsible User-Specific Recommendation. ACM Transactions on Intelligent Systems and Technology (2024). Just Accepted.
  7. Human-level play in the game of Diplomacy by combining language models with strategic reasoning. Science 378, 6624 (2022), 1067–1074.
  8. Learning to Exploit Long-term Relational Dependencies in Knowledge Graphs. In Proceedings of the 36th International Conference on Machine Learning. 2505–2514.
  9. What Can Knowledge Bring to Machine Learning?—A Survey of Low-shot Learning for Structured Data. 13, 3, Article 48 (2022), 45 pages.
  10. How Can We Know What Language Models Know? Transactions of the Association for Computational Linguistics 8 (2020), 423–438.
  11. Challenges and applications of large language models. arXiv:2307.10169 (2023).
  12. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, Vol. 1. 2.
  13. Shima Khoshraftar and Aijun An. 2024. A Survey on Graph Representation Learning Methods. ACM Transactions on Intelligent Systems and Technology 15, 1, Article 19 (2024), 55 pages.
  14. Multi-Task Learning for Knowledge Graph Completion with Pre-trained Language Models. In Proceedings of the 28th International Conference on Computational Linguistics. 1737–1743.
  15. Dbpedia–a large-scale, multilingual knowledge base extracted from wikipedia. Semantic web 6, 2 (2015), 167–195.
  16. Semi-supervised Entity Alignment via Joint Knowledge Embedding Model and Cross-graph Model. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2723–2732.
  17. KGNN: Knowledge Graph Neural Network for Drug-Drug Interaction Prediction.. In Proceedings of the 29th International Joint Conference on Artificial Intelligence. 2739–2745.
  18. Multi-modal Contrastive Representation Learning for Entity Alignment. In Proceedings of the 29th International Conference on Computational Linguistics. 2572–2584.
  19. Generated Knowledge Prompting for Commonsense Reasoning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 3154–3169.
  20. Evolving Knowledge Graph Representation Learning with Multiple Attention Strategies for Citation Recommendation System. ACM Transactions on Intelligent Systems and Technology (2024). Just Accepted.
  21. K-bert: Enabling language representation with knowledge graph. In Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2901–2908.
  22. UrbanKG: An Urban Knowledge Graph System. ACM Transactions On Intelligent Systems and Technology 14, 4, Article 60 (2023), 25 pages.
  23. STAR: Improving Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models. In NeurIPS 2023 Workshop on Synthetic Data Generation with Generative AI.
  24. Yago3: A knowledge base from multilingual wikipedias. In Proceedings of the 7th Biennial Conference on Innovative Data Systems Research.
  25. Are Negative Samples Necessary in Entity Alignment? An Approach with High Performance, Scalability and Robustness. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 1263–1273.
  26. MRAEA: An Efficient and Robust Entity Alignment Approach for Cross-Lingual Knowledge Graph. In Proceedings of the 13th International Conference on Web Search and Data Mining. 420–428.
  27. Relational Reflection Entity Alignment. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 1095–1104.
  28. Sound and Music Recommendation with Knowledge Graphs. ACM Transactions On Intelligent Systems and Technology 8, 2, Article 21 (2016), 21 pages.
  29. A Plug-and-Play Method for Controlled Text Generation. In Findings of the Association for Computational Linguistics: EMNLP 2021. 3973–3997.
  30. Embedding-Based Entity Alignment Using Relation Structural Similarity. In 2020 IEEE International Conference on Knowledge Graph. 123–130.
  31. Explicit Knowledge Graph Reasoning for Conversational Recommendation. ACM Transactions On Intelligent Systems and Technology (2023). Just Accepted.
  32. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 4222–4235.
  33. Cross-Lingual Entity Alignment via Joint Attribute-Preserving Embedding. In Proceedings of the 16th International Semantic Web Conference. 628–644.
  34. Bootstrapping Entity Alignment with Knowledge Graph Embedding. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 4396–4402.
  35. TransEdge: Translating Relation-Contextualized Embeddings for Knowledge Graphs. In Proceedings of the 18th International Semantic Web Conference. 612–629.
  36. Knowledge graph alignment network with gated multi-hop neighborhood aggregation. In Proceedings of the 34th AAAI Conference on Artificial Intelligence. 222–229.
  37. IntelliCode Compose: Code Generation Using Transformer. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1433–1443.
  38. Graph Attention Networks. In Proceedings of the 6th International Conference on Learning Representations.
  39. HGATE: Heterogeneous Graph Attention Auto-Encoders. IEEE Transactions on Knowledge and Data Engineering 35, 4 (2023), 3938–3951.
  40. Cross-lingual Knowledge Graph Alignment via Graph Convolutional Networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP). 349–357.
  41. Emergent abilities of large language models. arXiv:2206.07682 (2022).
  42. Empowering News Recommendation with Pre-Trained Language Models. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1652–1656.
  43. Relation-Aware Entity Alignment for Heterogeneous Knowledge Graphs. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 5278–5284.
  44. Jointly Learning Entity and Relation Representations for Entity Alignment. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 240–249.
  45. Neighborhood Matching Network for Entity Alignment. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 6477–6487.
  46. Cross-Modal Graph Attention Network for Entity Alignment. In Proceedings of the 31st ACM International Conference on Multimedia. 3715–3723.
  47. Cross-lingual Knowledge Graph Alignment via Graph Matching Neural Network. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 3156–3161.
  48. ChatGPT is not Enough: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling. arXiv:2306.11489 (2023).
  49. Collective Entity Alignment for Knowledge Fusion of Power Grid Dispatching Knowledge Graphs. IEEE/CAA Journal of Automatica Sinica 9, 11 (2022), 1990–2004.
  50. HackRL: Reinforcement learning with hierarchical attention for cross-graph knowledge fusion and collaborative reasoning. Knowledge-Based Systems 233 (2021), 107498.
  51. Learning Network-Invariant and Label-Discriminative Representations for Cross-Network Node Classification. In 2021 China Automation Congress. 5075–5080.
  52. QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 535–546.
  53. A Vectorized Relational Graph Convolutional Network for Multi-Relational Network Alignment. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 4135–4141.
  54. Large Language Models for Social Networks: Applications, Challenges, and Solutions. arXiv:2401.02575 (2024).
  55. Collective Entity Alignment via Adaptive Features. In IEEE 36th International Conference on Data Engineering. 1870–1873.
  56. Reinforcement Learning–Based Collective Entity Alignment with Adaptive Features. ACM Transactions on Information Systems 39, 3, Article 26 (2021).
  57. Degree-Aware Alignment for Entities in Tail. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 811–820.
  58. Deep multimodal fusion for semantic image segmentation: A survey. Image and Vision Computing 105 (2021), 104042.
  59. An experimental study of state-of-the-art entity alignment approaches. IEEE Transactions on Knowledge and Data Engineering 34, 6 (2020), 2610–2625.
  60. Graph Neural Networks: Taxonomy, Advances, and Trends. ACM Transactions on Intelligent Systems and Technology 13, 1, Article 15 (2022), 54 pages.
  61. Iterative Entity Alignment via Joint Knowledge Embeddings. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 4258–4264.
  62. RAGA: Relation-Aware Graph Attention Networks for Global Entity Alignment. In Advances in Knowledge Discovery and Data Mining. 501–513.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Linyao Yang (5 papers)
  2. Hongyang Chen (61 papers)
  3. Xiao Wang (507 papers)
  4. Jing Yang (320 papers)
  5. Fei-Yue Wang (71 papers)
  6. Han Liu (340 papers)
Citations (2)