Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
113 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
24 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
3 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
2000 character limit reached

EnriCo: Enriched Representation and Globally Constrained Inference for Entity and Relation Extraction (2404.12493v1)

Published 18 Apr 2024 in cs.CL and cs.AI

Abstract: Joint entity and relation extraction plays a pivotal role in various applications, notably in the construction of knowledge graphs. Despite recent progress, existing approaches often fall short in two key aspects: richness of representation and coherence in output structure. These models often rely on handcrafted heuristics for computing entity and relation representations, potentially leading to loss of crucial information. Furthermore, they disregard task and/or dataset-specific constraints, resulting in output structures that lack coherence. In our work, we introduce EnriCo, which mitigates these shortcomings. Firstly, to foster rich and expressive representation, our model leverage attention mechanisms that allow both entities and relations to dynamically determine the pertinent information required for accurate extraction. Secondly, we introduce a series of decoding algorithms designed to infer the highest scoring solutions while adhering to task and dataset-specific constraints, thus promoting structured and coherent outputs. Our model demonstrates competitive performance compared to baselines when evaluated on Joint IE datasets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. Nguyen Bach and Sameer Badaskar. 2007. A review of relation extraction.
  2. Scibert: A pretrained language model for scientific text. In Conference on Empirical Methods in Natural Language Processing.
  3. Answer set programming at a glance. Communications of the ACM, 54:92 – 103.
  4. Sergey Brin. 1999. Extracting patterns and relations from the world wide web. In The World Wide Web and Databases, pages 172–183, Berlin, Heidelberg. Springer Berlin Heidelberg.
  5. Xavier Carreras and Lluís Màrquez. 2004. Introduction to the CoNLL-2004 shared task: Semantic role labeling. In Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004, pages 89–97, Boston, Massachusetts, USA. Association for Computational Linguistics.
  6. Jason P. C. Chiu and Eric Nichols. 2015. Named entity recognition with bidirectional lstm-cnns. Transactions of the Association for Computational Linguistics, 4:357–370.
  7. Bert: Pre-training of deep bidirectional transformers for language understanding. ArXiv, abs/1810.04805.
  8. Markus Eberts and Adrian Ulges. 2019. Span-based joint entity and relation extraction with transformer pre-training. ArXiv, abs/1909.07755.
  9. LasUIE: Unifying information extraction with latent adaptive structure-aware generative language model. In Advances in Neural Information Processing Systems.
  10. Attention sur les spans pour l’analyse syntaxique en constituants. In Actes de CORIA-TALN 2023. Actes de la 30e Conférence sur le Traitement Automatique des Langues Naturelles (TALN), volume 2 : travaux de recherche originaux – articles courts, pages 37–45, Paris, France. ATALA.
  11. Graphrel: Modeling text as relational graphs for joint entity and relation extraction. In Annual Meeting of the Association for Computational Linguistics.
  12. Clingo= asp+ control: Preliminary report. arXiv preprint arXiv:1405.3694.
  13. Categorical reparameterization with gumbel-softmax. In International Conference on Learning Representations.
  14. Span-based joint entity and relation extraction with attention-based span-specific and contextual semantic representations. In Proceedings of the 28th International Conference on Computational Linguistics, pages 88–99, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  15. Improving span representation by efficient span-level attention. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11184–11192, Singapore. Association for Computational Linguistics.
  16. Neural architectures for named entity recognition. In North American Chapter of the Association for Computational Linguistics.
  17. Albert: A lite bert for self-supervised learning of language representations. ArXiv, abs/1909.11942.
  18. Neural relation extraction with selective attention over instances. In Annual Meeting of the Association for Computational Linguistics.
  19. A joint neural model for information extraction with global features. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7999–8009, Online. Association for Computational Linguistics.
  20. Unified structure generation for universal information extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5755–5772, Dublin, Ireland. Association for Computational Linguistics.
  21. Multi-task identification of entities, relations, and coreferencefor scientific knowledge graph construction. In Proc. Conf. Empirical Methods Natural Language Process. (EMNLP).
  22. Joint entity and relation extraction based on table labeling using convolutional neural networks. In Proceedings of the Sixth Workshop on Structured Prediction for NLP, pages 11–21, Dublin, Ireland. Association for Computational Linguistics.
  23. David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Lingvisticae Investigationes, 30:3–26.
  24. A review of relational machine learning for knowledge graphs. Proceedings of the IEEE, 104(1):11–33.
  25. OpenAI. 2023. Gpt-4 technical report.
  26. Structured prediction as translation between augmented natural languages. In International Conference on Learning Representations.
  27. Dan Roth and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004, pages 1–8, Boston, Massachusetts, USA. Association for Computational Linguistics.
  28. Joint entity and relation extraction with set prediction networks. IEEE transactions on neural networks and learning systems, PP.
  29. Progressive multi-task learning with controlled information flow for joint entity and relation extraction. Proceedings of the AAAI Conference on Artificial Intelligence, 35(15):13851–13859.
  30. Ranking with ordered weighted pairwise classification. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML ’09, page 1057–1064, New York, NY, USA. Association for Computing Machinery.
  31. Attention is all you need. In NIPS.
  32. Graph attention networks. ArXiv, abs/1710.10903.
  33. Entity, relation, and event extraction with contextualized span representations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5784–5789, Hong Kong, China. Association for Computational Linguistics.
  34. Revisiting relation extraction in the era of large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15566–15589, Toronto, Canada. Association for Computational Linguistics.
  35. Ace 2005 multilingual training corpus.
  36. Jue Wang and Wei Lu. 2020. Two are better than one: Joint entity and relation extraction with table-sequence encoders. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1706–1721, Online. Association for Computational Linguistics.
  37. UniRE: A unified label space for entity relation extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 220–231, Online. Association for Computational Linguistics.
  38. Adversarial training for relation extraction. In Conference on Empirical Methods in Natural Language Processing.
  39. UTC-IE: A unified token-pair classification architecture for information extraction. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4096–4122, Toronto, Canada. Association for Computational Linguistics.
  40. A partition filter network for joint entity and relation extraction. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 185–197, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  41. Packed levitated marker for entity and relation extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4904–4917, Dublin, Ireland. Association for Computational Linguistics.
  42. GNNer: Reducing overlapping in span-based NER using graph neural networks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 97–103, Dublin, Ireland. Association for Computational Linguistics.
  43. Named entity recognition as structured span prediction. In Proceedings of the Workshop on Unimodal and Multimodal Induction of Linguistic Structures (UM-IoS), pages 1–10, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  44. An autoregressive text-to-graph framework for joint entity and relation extraction. Proceedings of the AAAI Conference on Artificial Intelligence, 38(17):19477–19487.
  45. Kernel methods for relation extraction. In Journal of machine learning research.
  46. A unified multi-task learning framework for joint extraction of entities and relations. Proceedings of the AAAI Conference on Artificial Intelligence, 35(16):14524–14531.
  47. Zexuan Zhong and Danqi Chen. 2021. A frustratingly easy approach for entity and relation extraction. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 50–61, Online. Association for Computational Linguistics.
  48. Deep span representations for named entity recognition. In Findings of the Association for Computational Linguistics: ACL 2023, pages 10565–10582, Toronto, Canada. Association for Computational Linguistics.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now