Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

FSUIE: A Novel Fuzzy Span Mechanism for Universal Information Extraction (2306.14913v1)

Published 19 Jun 2023 in cs.CL and cs.AI

Abstract: Universal Information Extraction (UIE) has been introduced as a unified framework for various Information Extraction (IE) tasks and has achieved widespread success. Despite this, UIE models have limitations. For example, they rely heavily on span boundaries in the data during training, which does not reflect the reality of span annotation challenges. Slight adjustments to positions can also meet requirements. Additionally, UIE models lack attention to the limited span length feature in IE. To address these deficiencies, we propose the Fuzzy Span Universal Information Extraction (FSUIE) framework. Specifically, our contribution consists of two concepts: fuzzy span loss and fuzzy span attention. Our experimental results on a series of main IE tasks show significant improvement compared to the baseline, especially in terms of fast convergence and strong performance with small amounts of data and training epochs. These results demonstrate the effectiveness and generalization of FSUIE in different tasks, settings, and scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Massively multilingual neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3874--3884, Minneapolis, Minnesota. Association for Computational Linguistics.
  2. Douglas E. Appelt and Boyan Onyshkevych. 1998. The common pattern specification language. In TIPSTER TEXT PROGRAM PHASE III: Proceedings of a Workshop held at Baltimore, Maryland, October 13-15, 1998, pages 23--30, Baltimore, Maryland, USA. Association for Computational Linguistics.
  3. Massively multilingual neural machine translation in the wild: Findings and challenges. CoRR, abs/1907.05019.
  4. A span-level bidirectional network for aspect sentiment triplet extraction. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP).
  5. Generating long sequences with sparse transformers. CoRR, abs/1904.10509.
  6. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440--8451, Online. Association for Computational Linguistics.
  7. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171--4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  8. Markus Eberts and Adrian Ulges. 2019. Span-based joint entity and relation extraction with transformer pre-training. CoRR, abs/1909.07755.
  9. Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. J. Biomed. Informatics, 45(5):885--892.
  10. Jcbie: a joint continual learning neural network for biomedical information extraction. BMC bioinformatics, 23(1):1--20.
  11. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328--339, Melbourne, Australia. Association for Computational Linguistics.
  12. Boningknife: Joint entity mention detection and typing for nested NER via prior boundary knowledge. CoRR, abs/2107.09429.
  13. Seeking common but distinguishing difference, a joint aspect-based sentiment analysis model. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3910--3922, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  14. A supervised machine learning approach for event-event relation identification. In Proceedings of the 24th Pacific Asia Conference on Language, Information and Computation, pages 447--454, Tohoku University, Sendai, Japan. Institute of Digital Enhancement of Cognitive Processing, Waseda University.
  15. Joint biomedical entity and relation extraction with knowledge-enhanced collective inference. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6248--6260, Online. Association for Computational Linguistics.
  16. An unsupervised multiple-task and multiple-teacher model for cross-lingual named entity recognition. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 170--179, Dublin, Ireland. Association for Computational Linguistics.
  17. Joint learning of POS and dependencies for multilingual Universal Dependency parsing. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 65--73, Brussels, Belgium. Association for Computational Linguistics.
  18. Text2Event: Controllable sequence-to-structure generation for end-to-end event extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2795--2806, Online. Association for Computational Linguistics.
  19. Unified structure generation for universal information extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5755--5772, Dublin, Ireland. Association for Computational Linguistics.
  20. Named entity recognition and relation extraction using enhanced table filling by contextualized representations. CoRR, abs/2010.07522.
  21. A joint training dual-mrc framework for aspect based sentiment analysis. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 13543--13551. AAAI Press.
  22. Reasoning with latent structure refinement for document-level relation extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1546--1557, Online. Association for Computational Linguistics.
  23. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227--2237, New Orleans, Louisiana. Association for Computational Linguistics.
  24. Robust distant supervision relation extraction via deep reinforcement learning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2137--2147, Melbourne, Australia. Association for Computational Linguistics.
  25. Subsequence based deep active learning for named entity recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4310--4321, Online. Association for Computational Linguistics.
  26. Joint entity and relation extraction from scientific documents: Role of linguistic information and entity types.
  27. Global pointer: Novel efficient span-based approach for named entity recognition. CoRR, abs/2208.03054.
  28. Adaptive attention span in transformers. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 331--335, Florence, Italy. Association for Computational Linguistics.
  29. Cross-lingual universal dependency parsing only from one monolingual treebank. arXiv preprint arXiv:2012.13163.
  30. Boundary enhanced neural span classification for nested named entity recognition. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 9016--9023. AAAI Press.
  31. A machine learning approach to information extraction. In Computational Linguistics and Intelligent Text Processing, 6th International Conference, CICLing 2005, Mexico City, Mexico, February 13-19, 2005, Proceedings, volume 3406 of Lecture Notes in Computer Science, pages 539--547. Springer.
  32. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998--6008.
  33. Jue Wang and Wei Lu. 2020. Two are better than one: Joint entity and relation extraction with table-sequence encoders. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1706--1721, Online. Association for Computational Linguistics.
  34. Wenya Wang and Sinno Jialin Pan. 2020. Integrating deep learning with logic fusion for information extraction. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 9225--9232. AAAI Press.
  35. Learning span-level interactions for aspect sentiment triplet extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4755--4766, Online. Association for Computational Linguistics.
  36. Position-aware tagging for aspect sentiment triplet extraction. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2339--2349, Online. Association for Computational Linguistics.
  37. A unified generative framework for various NER subtasks. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 5808--5822, Online. Association for Computational Linguistics.
  38. Nested named entity recognition as corpus aware holistic structure parsing. In Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12-17, 2022, pages 2472--2482. International Committee on Computational Linguistics.
  39. Packed levitated marker for entity and relation extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4904--4917, Dublin, Ireland. Association for Computational Linguistics.
  40. Fusing heterogeneous factors with triaffine mechanism for nested named entity recognition. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3174--3186, Dublin, Ireland. Association for Computational Linguistics.
  41. Big bird: Transformers for longer sequences. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
  42. Towards generative aspect-based sentiment analysis. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 504--510, Online. Association for Computational Linguistics.
  43. Modeling named entity embedding distribution into hypersphere. CoRR, abs/1909.01065.
  44. Enwei Zhu and Jinpeng Li. 2022. Boundary smoothing for named entity recognition. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7096--7108, Dublin, Ireland. Association for Computational Linguistics.
  45. Long-range sequence modeling with predictable sparse attention. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 234--243, Dublin, Ireland. Association for Computational Linguistics.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Tianshuo Peng (10 papers)
  2. Zuchao Li (76 papers)
  3. Lefei Zhang (64 papers)
  4. Bo Du (264 papers)
  5. Hai Zhao (227 papers)
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now