Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction (2401.01326v2)

Published 2 Jan 2024 in cs.CL, cs.AI, and cs.LG

Abstract: In this paper, we propose a novel method for joint entity and relation extraction from unstructured text by framing it as a conditional sequence generation problem. In contrast to conventional generative information extraction models that are left-to-right token-level generators, our approach is \textit{span-based}. It generates a linearized graph where nodes represent text spans and edges represent relation triplets. Our method employs a transformer encoder-decoder architecture with pointing mechanism on a dynamic vocabulary of spans and relation types. Our model can capture the structural characteristics and boundaries of entities and relations through span representations while simultaneously grounding the generated output in the original text thanks to the pointing mechanism. Evaluation on benchmark datasets validates the effectiveness of our approach, demonstrating competitive results. Code is available at https://github.com/urchade/ATG.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. A Review of Relation Extraction.
  2. SciBERT: A Pretrained Language Model for Scientific Text. In Conference on Empirical Methods in Natural Language Processing.
  3. Brin, S. 1999. Extracting Patterns and Relations from the World Wide Web. In Atzeni, P.; Mendelzon, A.; and Mecca, G., eds., The World Wide Web and Databases, 172–183. Berlin, Heidelberg: Springer Berlin Heidelberg. ISBN 978-3-540-48909-2.
  4. Autoregressive Entity Retrieval. In International Conference on Learning Representations.
  5. Introduction to the CoNLL-2004 Shared Task: Semantic Role Labeling. In Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004, 89–97. Boston, Massachusetts, USA.
  6. Reading Wikipedia to Answer Open-Domain Questions. In Barzilay, R.; and Kan, M.-Y., eds., Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
  7. Pix2seq: A Language Modeling Framework for Object Detection. In International Conference on Learning Representations.
  8. Named Entity Recognition with Bidirectional LSTM-CNNs. Transactions of the Association for Computational Linguistics, 4: 357–370.
  9. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics., 4171–4186. Minneapolis, Minnesota.
  10. Span-based Joint Entity and Relation Extraction with Transformer Pre-training. ArXiv, abs/1909.07755.
  11. LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model. In Oh, A. H.; Agarwal, A.; Belgrave, D.; and Cho, K., eds., Advances in Neural Information Processing Systems.
  12. A sequence-to-sequence approach for document-level relation extraction. In Workshop on Biomedical Natural Language Processing.
  13. DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing. ArXiv, abs/2111.09543.
  14. The Curious Case of Neural Text Degeneration. ArXiv, abs/1904.09751.
  15. Span-based Joint Entity and Relation Extraction with Attention-based Span-specific and Contextual Semantic Representations. In Proceedings of the 28th International Conference on Computational Linguistics, 88–99. Barcelona, Spain (Online): International Committee on Computational Linguistics.
  16. Characterizing and addressing the issue of oversmoothing in neural autoregressive sequence modeling. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).
  17. Neural Architectures for Named Entity Recognition. In North American Chapter of the Association for Computational Linguistics.
  18. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 7871–7880.
  19. Document-Level Event Argument Extraction by Conditional Generation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 894–908.
  20. Neural Relation Extraction with Selective Attention over Instances. In Annual Meeting of the Association for Computational Linguistics.
  21. Autoregressive Structured Prediction with Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2022, 993–1005. Abu Dhabi, United Arab Emirates.
  22. Decoupled Weight Decay Regularization. In International Conference on Learning Representations.
  23. Unified Structure Generation for Universal Information Extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 5755–5772. Dublin, Ireland: Association for Computational Linguistics.
  24. Multi-Task Identification of Entities, Relations, and Coreferencefor Scientific Knowledge Graph Construction. In Proc. Conf. Empirical Methods Natural Language Process. (EMNLP).
  25. Joint Entity and Relation Extraction Based on Table Labeling Using Convolutional Neural Networks. In Proceedings of the Sixth Workshop on Structured Prediction for NLP, 11–21.
  26. Correcting Length Bias in Neural Machine Translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, 212–223. Brussels, Belgium.
  27. A survey of named entity recognition and classification. Lingvisticae Investigationes, 30: 3–26.
  28. Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extraction. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, New York, NY, USA, February 7-12, 2020, 8528–8535. AAAI Press.
  29. Structured Prediction as Translation between Augmented Natural Languages. In International Conference on Learning Representations.
  30. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. ArXiv, abs/1910.10683.
  31. HySPA: Hybrid Span Generation for Scalable Text-to-Graph Extraction. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 4066–4078.
  32. A Linear Programming Formulation for Global Inference in Natural Language Tasks. In Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004, 1–8. Boston, Massachusetts, USA.
  33. OneRel: Joint Entity and Relation Extraction with One Module in One Step. In AAAI Conference on Artificial Intelligence.
  34. Joint Entity and Relation Extraction with Set Prediction Networks. IEEE transactions on neural networks and learning systems, PP.
  35. Progressive Multi-task Learning with Controlled Information Flow for Joint Entity and Relation Extraction. Proceedings of the AAAI Conference on Artificial Intelligence, 35(15): 13851–13859.
  36. Let’s Stop Incorrect Comparisons in End-to-end Relation Extraction! arXiv:2009.10684.
  37. Attention is All you Need. In NIPS.
  38. Pointer Networks. In Cortes, C.; Lawrence, N.; Lee, D.; Sugiyama, M.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc.
  39. Entity, Relation, and Event Extraction with Contextualized Span Representations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).
  40. ACE 2005 Multilingual Training Corpus.
  41. Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).
  42. UniRE: A Unified Label Space for Entity Relation Extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 220–231.
  43. Adversarial Training for Relation Extraction. In Conference on Empirical Methods in Natural Language Processing.
  44. Reducing Length Bias in Scoring Neural Machine Translation via a Causal Inference Method. In Proceedings of the 20th Chinese National Conference on Computational Linguistics, 874–885. Huhhot, China: Chinese Information Processing Society of China.
  45. A Partition Filter Network for Joint Entity and Relation Extraction. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 185–197. Association for Computational Linguistics.
  46. Packed Levitated Marker for Entity and Relation Extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 4904–4917. Dublin, Ireland: Association for Computational Linguistics.
  47. Generative Knowledge Graph Construction: A Review. In Goldberg, Y.; Kozareva, Z.; and Zhang, Y., eds., Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 1–17. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics.
  48. Kernel Methods for Relation Extraction. In Journal of machine learning research.
  49. A Unified Multi-Task Learning Framework for Joint Extraction of Entities and Relations. Proceedings of the AAAI Conference on Artificial Intelligence, 35(16): 14524–14531.
  50. A Frustratingly Easy Approach for Entity and Relation Extraction. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets