Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

ChatEL: Entity Linking with Chatbots (2402.14858v1)

Published 20 Feb 2024 in cs.CL and cs.AI

Abstract: Entity Linking (EL) is an essential and challenging task in natural language processing that seeks to link some text representing an entity within a document or sentence with its corresponding entry in a dictionary or knowledge base. Most existing approaches focus on creating elaborate contextual models that look for clues the words surrounding the entity-text to help solve the linking problem. Although these fine-tuned LLMs tend to work, they can be unwieldy, difficult to train, and do not transfer well to other domains. Fortunately, LLMs like GPT provide a highly-advanced solution to the problems inherent in EL models, but simply naive prompts to LLMs do not work well. In the present work, we define ChatEL, which is a three-step framework to prompt LLMs to return accurate results. Overall the ChatEL framework improves the average F1 performance across 10 datasets by more than 2%. Finally, a thorough error analysis shows many instances with the ground truth labels were actually incorrect, and the labels predicted by ChatEL were actually correct. This indicates that the quantitative results presented in this paper may be a conservative estimate of the actual performance. All data and code are available as an open-source package on GitHub at https://github.com/yifding/In_Context_EL.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. FLAIR: An easy-to-use framework for state-of-the-art NLP. In NAACL 2019, 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pages 54–59.
  2. ReFinED: An efficient zero-shot-capable approach to end-to-end entity linking. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track, pages 209–220, Hybrid: Seattle, Washington + Online. Association for Computational Linguistics.
  3. ExtEnD: Extractive entity disambiguation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2478–2488, Dublin, Ireland. Association for Computational Linguistics.
  4. Zero-shot entity linking with less data. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1681–1697, Seattle, United States. Association for Computational Linguistics.
  5. Reddit entity linking dataset. Information Processing & Management, 58(3):102479.
  6. Rodney A Brooks. 1981. Symbolic reasoning among 3-d models and 2-d images. Artificial Intelligence, 17(1-3):285–348.
  7. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc.
  8. Autoregressive entity retrieval. In International Conference on Learning Representations.
  9. Harrison Chase. 2022. LangChain.
  10. Learning in-context learning for named entity recognition. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13661–13675, Toronto, Canada. Association for Computational Linguistics.
  11. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.
  12. Silviu Cucerzan. 2007. Large-scale named entity disambiguation based on Wikipedia data. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 708–716, Prague, Czech Republic. Association for Computational Linguistics.
  13. Template-based named entity recognition using BART. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1835–1845, Online. Association for Computational Linguistics.
  14. DARPA. 2018. Ai next campaign. DARPA Website.
  15. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  16. Prompt-learning for fine-grained entity typing. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6888–6901, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  17. Posthoc verification and the fallibility of the ground truth. In Proceedings of the First Workshop on Dynamic Adversarial Data Collection, pages 23–29, Seattle, WA. Association for Computational Linguistics.
  18. Robin Emsley. 2023. Chatgpt: these are not hallucinations–they’re fabrications and falsifications. Schizophrenia, 9(1):52.
  19. Joint entity linking with deep reinforcement learning. In The World Wide Web Conference, WWW ’19, page 438–447, New York, NY, USA. Association for Computing Machinery.
  20. Lasuie: Unifying information extraction with latent adaptive structure-aware generative language model. Advances in Neural Information Processing Systems, 35:15460–15475.
  21. Facc1: Freebase annotation of clueweb corpora, version 1 (release date 2013-06-26, format version 1, correction level 0).
  22. Octavian-Eugen Ganea and Thomas Hofmann. 2017. Deep joint entity disambiguation with local neural attention. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2619–2629, Copenhagen, Denmark. Association for Computational Linguistics.
  23. Yes but.. can chatgpt identify entities in historical documents? arXiv preprint arXiv:2303.17322.
  24. Zhaochen Guo and Denilson Barbosa. 2018. Robust named entity disambiguation with random walks. Semantic Web, 9(4):459–479.
  25. Pascal Hitzler and Md Kamruzzaman Sarker. 2022. Neuro-symbolic artificial intelligence: The state of the art. IOS Press.
  26. Kore: Keyphrase overlap relatedness for entity disambiguation. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM ’12, page 545–554, New York, NY, USA. Association for Computing Machinery.
  27. Robust disambiguation of named entities in text. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 782–792, Edinburgh, Scotland, UK. Association for Computational Linguistics.
  28. Graph neural entity disambiguation. Knowledge-Based Systems, 195:105620.
  29. End-to-end neural entity linking. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 519–529, Brussels, Belgium. Association for Computational Linguistics.
  30. Phong Le and Ivan Titov. 2018. Improving entity linking by modeling latent relations between mentions. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1595–1604, Melbourne, Australia. Association for Computational Linguistics.
  31. Zero-shot entity linking by reading entity descriptions. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3449–3460, Florence, Italy. Association for Computational Linguistics.
  32. Unified structure generation for universal information extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5755–5772, Dublin, Ireland. Association for Computational Linguistics.
  33. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc.
  34. David Milne and Ian H. Witten. 2008. Learning to link with wikipedia. In Proceedings of the 17th ACM Conference on Information and Knowledge Management, CIKM ’08, page 509–518, New York, NY, USA. Association for Computing Machinery.
  35. Using citations to generate surveys of scientific paradigms. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 584–592, Boulder, Colorado. Association for Computational Linguistics.
  36. Local-global video-text interactions for temporal grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10810–10819.
  37. Open knowledge extraction challenge. In Semantic Web Evaluation Challenges, pages 3–15, Cham. Springer International Publishing.
  38. The second open knowledge extraction challenge. In Semantic Web Challenges, pages 3–16, Cham. Springer International Publishing.
  39. OpenAI. 2023. Gpt-4 technical report.
  40. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744.
  41. Pair-linking for collective entity disambiguation: Two could be better than all. IEEE Transactions on Knowledge and Data Engineering, 31(7):1383–1396.
  42. Haystack: the end-to-end NLP framework for pragmatic builders.
  43. Is chatgpt a general-purpose natural language processing task solver? arXiv preprint arXiv:2302.06476.
  44. Improving language understanding by generative pre-training. OpenAI Blog.
  45. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  46. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(1).
  47. Local and global algorithms for disambiguation to Wikipedia. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1375–1384, Portland, Oregon, USA. Association for Computational Linguistics.
  48. N33{}^{3}start_FLOATSUPERSCRIPT 3 end_FLOATSUPERSCRIPT - a collection of datasets for named entity recognition and disambiguation in the NLP interchange format. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 3529–3533, Reykjavik, Iceland. European Language Resources Association (ELRA).
  49. PromptNER: Prompt locating and typing for named entity recognition. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12492–12507, Toronto, Canada. Association for Computational Linguistics.
  50. Head-to-tail: How knowledgeable are large language models (llm)? aka will llms replace knowledge graphs? arXiv preprint arXiv:2308.10168.
  51. Named Entity Recognition for Entity Linking: What works and what’s next. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2584–2596, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  52. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  53. Rel: An entity linker standing on the shoulders of giants. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’20, page 2197–2200, New York, NY, USA. Association for Computing Machinery.
  54. Gpt-ner: Named entity recognition via large language models.
  55. CrossWeigh: Training named entity tagger from imperfect annotations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5154–5163, Hong Kong, China. Association for Computational Linguistics.
  56. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824–24837. Curran Associates, Inc.
  57. Scalable zero-shot entity linking with dense entity retrieval. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6397–6407, Online. Association for Computational Linguistics.
  58. Exploring the limits of chatgpt for query or aspect-based text summarization.
  59. Jaket: Joint pre-training of knowledge graph and language understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11630–11638.
  60. Generate rather than retrieve: Large language models are strong context generators. In The Eleventh International Conference on Learning Representations.
  61. Validating label consistency in ner data annotation. In Proceedings of the 2nd Workshop on Evaluation and Comparison of NLP Systems, pages 11–15.
  62. EntQA: Entity linking as question answering. In International Conference on Learning Representations.
  63. Wenxuan Zhou and Muhao Chen. 2021. Learning from noisy labels for entity-centric information extraction. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5381–5392, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Yifan Ding (44 papers)
  2. Qingkai Zeng (28 papers)
  3. Tim Weninger (67 papers)
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now