Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Enhancing Low-Resource Relation Representations through Multi-View Decoupling (2312.17267v4)

Published 26 Dec 2023 in cs.CL and cs.AI

Abstract: Recently, prompt-tuning with pre-trained LLMs (PLMs) has demonstrated the significantly enhancing ability of relation extraction (RE) tasks. However, in low-resource scenarios, where the available training data is scarce, previous prompt-based methods may still perform poorly for prompt-based representation learning due to a superficial understanding of the relation. To this end, we highlight the importance of learning high-quality relation representation in low-resource scenarios for RE, and propose a novel prompt-based relation representation method, named MVRE (\underline{M}ulti-\underline{V}iew \underline{R}elation \underline{E}xtraction), to better leverage the capacity of PLMs to improve the performance of RE within the low-resource prompt-tuning paradigm. Specifically, MVRE decouples each relation into different perspectives to encompass multi-view relation representations for maximizing the likelihood during relation inference. Furthermore, we also design a Global-Local loss and a Dynamic-Initialization method for better alignment of the multi-view relation-representing virtual words, containing the semantics of relation labels during the optimization learning process and initialization. Extensive experiments on three benchmark datasets show that our method can achieve state-of-the-art in low-resource settings.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 1558–1569.
  2. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877–1901.
  3. Relation Extraction as Open-book Examination: Retrieval-enhanced Prompt Tuning. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2443–2448.
  4. Knowprompt: Knowledge-aware prompt-tuning with synergistic optimization for relation extraction. In Proceedings of the ACM Web conference 2022, 2778–2788.
  5. HacRED: A large-scale relation extraction dataset toward hard cases in practical applications. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 2819–2831.
  6. Making Pre-trained Language Models Better Few-shot Learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 3816–3830.
  7. SimCSE: Simple Contrastive Learning of Sentence Embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 6894–6910.
  8. PPT: Pre-trained Prompt Tuning for Few-shot Learning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 8410–8423.
  9. Delving deep into regularity: a simple but effective method for Chinese named entity recognition. arXiv preprint arXiv:2204.05544.
  10. Ptr: Prompt tuning with rules for text classification. AI Open, 3: 182–192.
  11. SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals. ACL 2010, 33.
  12. FPT: Improving Prompt Tuning Efficiency via Progressive Training. In Findings of the Association for Computational Linguistics: EMNLP 2022, 6877–6887.
  13. Instance-aware prompt learning for language understanding and generation. ACM Transactions on Asian and Low-Resource Language Information Processing.
  14. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of NAACL-HLT, 4171–4186.
  15. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9): 1–35.
  16. Knowledge diffusion for neural dialogue generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1489–1498.
  17. Summarization as Indirect Supervision for Relation Extraction. In Findings of the Association for Computational Linguistics: EMNLP 2022, 6575–6594.
  18. MIRACLE: Towards Personalized Dialogue Generation with Latent-Space Multiple Personal Attribute Control. arXiv preprint arXiv:2310.18342.
  19. Passage retrieval for outside-knowledge visual question answering. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 1753–1757.
  20. Distantly-supervised named entity recognition with adaptive teacher learning and fine-grained student ensemble. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 13501–13509.
  21. Improving language understanding by generative pre-training.
  22. Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification. In Proceedings of the 28th International Conference on Computational Linguistics, 5569–5578.
  23. Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 255–269.
  24. Matching the Blanks: Distributional Similarity for Relation Learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2895–2905.
  25. Automatic Multi-Label Prompting: Simple and Interpretable Few-Shot Classification. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 5483–5492.
  26. Gdpnet: Refining latent multi-view graph for relation extraction. In Proceedings of the AAAI Conference on Artificial Intelligence, 14194–14202.
  27. Yang, Z. 2020. Biomedical information retrieval incorporating knowledge graph for explainable precision medicine. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2486–2486.
  28. QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 535–546.
  29. Coreferential Reasoning Learning for Language Representation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 7170–7186.
  30. Packed Levitated Marker for Entity and Relation Extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 4904–4917.
  31. Fusion-in-T5: Unifying Document Ranking Signals for Improved Information Retrieval. arXiv:2305.14685.
  32. Multi-View Document Representation Learning for Open-Domain Dense Retrieval. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 5990–6000.
  33. Position-aware attention and supervised data improve slot filling. In Conference on Empirical Methods in Natural Language Processing.
  34. An Improved Baseline for Sentence-level Relation Extraction. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing, 161–168.
  35. Efficient document-level event extraction via pseudo-trigger-aware pruned complete graph. arXiv preprint arXiv:2112.06013.
  36. Mirror: A Universal Framework for Various Information Extraction Tasks. arXiv preprint arXiv:2311.05419.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Chenghao Fan (7 papers)
  2. Wei Wei (425 papers)
  3. Xiaoye Qu (62 papers)
  4. Zhenyi Lu (9 papers)
  5. Wenfeng Xie (8 papers)
  6. Yu Cheng (354 papers)
  7. Dangyang Chen (20 papers)
Citations (2)
View on Semantic Scholar