Papers
Topics
Authors
Recent
Search
2000 character limit reached

TPN: Transferable Proto-Learning Network towards Few-shot Document-Level Relation Extraction

Published 1 Oct 2024 in cs.CL and cs.IR | (2410.00412v1)

Abstract: Few-shot document-level relation extraction suffers from poor performance due to the challenging cross-domain transferability of NOTA (none-of-the-above) relation representation. In this paper, we introduce a Transferable Proto-Learning Network (TPN) to address the challenging issue. It comprises three core components: Hybrid Encoder hierarchically encodes semantic content of input text combined with attention information to enhance the relation representations. As a plug-and-play module for Out-of-Domain (OOD) Detection, Transferable Proto-Learner computes NOTA prototype through an adaptive learnable block, effectively mitigating NOTA bias across various domains. Dynamic Weighting Calibrator detects relation-specific classification confidence, serving as dynamic weights to calibrate the NOTA-dominant loss function. Finally, to bolster the model's cross-domain performance, we complement it with virtual adversarial training (VAT). We conduct extensive experimental analyses on FREDo and ReFREDo, demonstrating the superiority of TPN. Compared to state-of-the-art methods, our approach achieves competitive performance with approximately half the parameter size. Data and code are available at https://github.com/EchoDreamer/TPN.

Authors (2)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. Y. Yao, D. Ye, P. Li, X. Han, Y. Lin, Z. Liu, Z. Liu, L. Huang, J. Zhou, and M. Sun, “DocRED: A large-scale document-level relation extraction dataset,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, Jul. 2019, pp. 764–777.
  2. W. Zhou, K. Huang, T. Ma, and J. Huang, “Document-level relation extraction with adaptive thresholding and localized context pooling,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 16, pp. 14 612–14 620, May 2021.
  3. S. Fan, S. Mo, and J. Niu, “Boosting document-level relation extraction by mining and injecting logical rules,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Abu Dhabi, United Arab Emirates, Dec. 2022, pp. 10 311–10 323.
  4. T. Hospedales, A. Antoniou, P. Micaelli, and A. Storkey, “Meta-learning in neural networks: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 9, pp. 5149–5169, 2022.
  5. N. Popovic and M. Färber, “Few-shot document-level relation extraction,” in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Seattle, United States, Jul. 2022, pp. 5733–5746.
  6. Y. Wang, R. Lou, K. Zhang, M. Y. Chen, and Y. Yang, “More: A metric learning based framework for open-domain relation extraction,” in ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 7698–7702.
  7. A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” 2019. [Online]. Available: https://arxiv.org/abs/1706.06083
  8. C. Zhu, Y. Cheng, Z. Gan, S. Sun, T. Goldstein, and J. Liu, “Freelb: Enhanced adversarial training for natural language understanding,” 2020. [Online]. Available: https://arxiv.org/abs/1909.11764
  9. S. Meng, X. Hu, A. Liu, S. Li, F. Ma, Y. Yang, and L. Wen, “RAPL: A relation-aware prototype learning approach for few-shot document-level relation extraction,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, Dec. 2023, pp. 5208–5226.
  10. X. Chen, N. Zhang, X. Xie, S. Deng, Y. Yao, C. Tan, F. Huang, L. Si, and H. Chen, “Knowprompt: Knowledge-aware prompt-tuning with synergistic optimization for relation extraction,” in Proceedings of the ACM Web Conference 2022, ser. WWW ’22, New York, NY, USA, 2022, p. 2778–2788.
  11. X. Zhu, Z. Kang, and B. Hui, “Fcds: Fusing constituency and dependency syntax into document-level relation extraction,” in Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, 2024.
  12. D. Sui, X. Zeng, Y. Chen, K. Liu, and J. Zhao, “Joint entity and relation extraction with set prediction networks,” IEEE Transactions on Neural Networks and Learning Systems, 2023.
  13. H. Liu, Z. Kang, L. Zhang, L. Tian, and F. Hua, “Document-level relation extraction with cross-sentence reasoning graph,” 2023. [Online]. Available: https://arxiv.org/abs/2303.03912
  14. Q. Tan, R. He, L. Bing, and H. T. Ng, “Document-level relation extraction with adaptive focal loss and knowledge distillation,” in Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, May 2022, pp. 1672–1681.
  15. W. Wang, B. Hu, Z. Peng, M. Zhong, Z. Zhang, Z. Liu, G. Zhang, and J. Zhou, “Garcia: Powering representations of long-tail query with multi–granularity contrastive learning,” in 2023 IEEE 39th International Conference on Data Engineering (ICDE), Los Alamitos, CA, USA, apr 2023, pp. 3182–3195.
  16. Q. Guo, Z. Kang, L. Tian, and Z. Chen, “Tiefake: Title-text similarity and emotion-aware fake news detection,” in 2023 International Joint Conference on Neural Networks (IJCNN).   IEEE, 2023, pp. 1–7.
  17. B. Xu, Q. Wang, Y. Lyu, Y. Zhu, and Z. Mao, “Entity structure within and throughout: Modeling mention dependencies for document-level relation extraction,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 16, pp. 14 149–14 157, May 2021.
  18. S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 10, pp. 1345–1359, 2010.
  19. Y. K. Chia, L. Bing, S. Poria, and L. Si, “RelationPrompt: Leveraging prompts to generate synthetic data for zero-shot relation triplet extraction,” in Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, May 2022, pp. 45–57.
  20. J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, “How transferable are features in deep neural networks?” in Advances in Neural Information Processing Systems, vol. 27, 2014.
  21. O. Sainz, O. Lopez de Lacalle, G. Labaka, A. Barrena, and E. Agirre, “Label verbalization and entailment for effective zero and few-shot relation extraction,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, Nov. 2021, pp. 1199–1212.
  22. S. Yang, Y. Zhang, G. Niu, Q. Zhao, and S. Pu, “Entity concept-enhanced few-shot relation extraction,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), Online, Aug. 2021, pp. 987–991.
  23. J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” in Advances in Neural Information Processing Systems, vol. 30, 2017.
  24. M. Wang, J. Zheng, F. Cai, T. Shao, and H. Chen, “DRK: Discriminative rule-based knowledge for relieving prediction confusions in few-shot relation extraction,” in Proceedings of the 29th International Conference on Computational Linguistics, Gyeongju, Republic of Korea, Oct. 2022, pp. 2129–2140.
  25. P. Zhang and W. Lu, “Better few-shot relation extraction with label prompt dropout,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Abu Dhabi, United Arab Emirates, Dec. 2022, pp. 6996–7006.
  26. S. Fort, J. Ren, and B. Lakshminarayanan, “Exploring the limits of out-of-distribution detection,” in Advances in Neural Information Processing Systems, vol. 34, 2021, pp. 7068–7081.
  27. O. Sabo, Y. Elazar, Y. Goldberg, and I. Dagan, “Revisiting few-shot relation classification: Evaluation data and classification schemes,” Transactions of the Association for Computational Linguistics, vol. 9, pp. 691–706, 2021.
  28. Y. Luan, L. He, M. Ostendorf, and H. Hajishirzi, “Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, Oct.-Nov. 2018, pp. 3219–3232.
  29. Q. Huang, S. Hao, Y. Ye, S. Zhu, Y. Feng, and D. Zhao, “Does recommend-revise produce reliable annotations? an analysis on missing instances in DocRED,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Dublin, Ireland, May 2022, pp. 6241–6252.
  30. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, Jun. 2019, pp. 4171–4186.
  31. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush, “Transformers: State-of-the-art natural language processing,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Online, Oct. 2020, pp. 38–45.
  32. J. Bjorck, K. Q. Weinberger, and C. Gomes, “Understanding decoupled and early weight decay,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 8, pp. 6777–6785, May 2021.
  33. P. Goyal, P. Dollár, R. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He, “Accurate, large minibatch sgd: Training imagenet in 1 hour,” 2018. [Online]. Available: https://arxiv.org/abs/1706.02677
  34. J. Han, B. Cheng, and W. Lu, “Exploring task difficulty for few-shot relation extraction,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, Nov. 2021, pp. 2605–2616.
  35. D. Wang, S. Wu, X. Zhang, and Z. Feng, “Multi-relation identification for few-shot document-level relation extraction,” in International Conference on Artificial Neural Networks.   Springer, 2023, pp. 52–64.
  36. L. van der Maaten and G. E. Hinton, “Visualizing data using t-sne,” Journal of Machine Learning Research, vol. 9, pp. 2579–2605, 2008.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up