Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 175 tok/s
Gemini 2.5 Pro 49 tok/s Pro
GPT-5 Medium 27 tok/s Pro
GPT-5 High 28 tok/s Pro
GPT-4o 67 tok/s Pro
Kimi K2 179 tok/s Pro
GPT OSS 120B 442 tok/s Pro
Claude Sonnet 4.5 35 tok/s Pro
2000 character limit reached

On the use of Silver Standard Data for Zero-shot Classification Tasks in Information Extraction (2402.18061v2)

Published 28 Feb 2024 in cs.CL and cs.AI

Abstract: The superior performance of supervised classification methods in the information extraction (IE) area heavily relies on a large amount of gold standard data. Recent zero-shot classification methods converted the task to other NLP tasks (e.g., textual entailment) and used off-the-shelf models of these NLP tasks to directly perform inference on the test data without using a large amount of IE annotation data. A potentially valuable by-product of these methods is the large-scale silver standard data, i.e., pseudo-labeled data by the off-the-shelf models of other NLP tasks. However, there is no further investigation into the use of these data. In this paper, we propose a new framework, Clean-LaVe, which aims to utilize silver standard data to enhance the zero-shot performance. Clean-LaVe includes four phases: (1) Obtaining silver data; (2) Identifying relatively clean data from silver data; (3) Finetuning the off-the-shelf model using clean data; (4) Inference on the test data. The experimental results show that Clean-LaVe can outperform the baseline by 5% and 6% on TACRED and Wiki80 dataset in the zero-shot relation classification task, and by 3%-7% on Smile (Korean and Polish) in the zero-shot cross-lingual relation classification task, and by 8% on ACE05-E+ in the zero-shot event argument classification task. The code is share in https://github.com/wjw136/Clean_LaVe.git.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. Open AI. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Görkem Algan and Ilkay Ulusoy. 2021. Image classification with deep learning in the presence of noisy labels: A survey. Knowledge Based System, 215:106771.
  3. Unsupervised label noise modeling and loss correction. In Proceedings of ICML, pages 312–321.
  4. A closer look at memorization in deep networks. In Proceedings of ICML, pages 233–242.
  5. On symmetric losses for learning from corrupted labels. In Proceedings of ICML, pages 961–970.
  6. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171–4186.
  7. Benoît Frénay and Michel Verleysen. 2013. Classification in the presence of label noise: a survey. IEEE transactions on neural networks and learning systems, 25(5):845–869.
  8. Robust loss functions under label noise for deep neural networks. In Proceedings of the AAAI, pages 1919–1925.
  9. Unsupervised relation extraction from language models using constrained cloze completion. In Findings of EMNLP, pages 1263–1276.
  10. A survey of label-noise representation learning: Past, present and future. arXiv preprint.
  11. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Proceedings of NeurIPS, pages 8536–8546.
  12. Deberta: Decoding-enhanced bert with disentangled attention. In Proceedings of ICLR.
  13. O2u-net: A simple noisy label detection approach for deep neural networks. In Proceedings of ICCV, pages 3326–3334.
  14. Multilingual generative language models for zero-shot cross-lingual event argument extraction. In Proceedings ACL, pages 4633–4646.
  15. Zero-shot transfer learning for event extraction. In Proceedings of ACL, pages 2160–2170.
  16. Uncertainty-aware learning against label noise on imbalanced datasets. In Proceedings of AAAI, volume 36, pages 6960–6969.
  17. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In Proceedings of ICML, pages 2304–2313.
  18. Instance-adaptive training with noise-robust losses against noisy labels. In Proceedings of EMNLP, pages 5647–5663.
  19. Nlnl: Negative learning for noisy labels. In Proceedings of ICCV, pages 101–110.
  20. Less Annotating, More Classifying – Addressing the Data Scarcity Issue of Supervised Machine Learning with Deep Transfer Learning and BERT - NLI. Preprint. Publisher: Open Science Framework.
  21. Zero-shot relation extraction via reading comprehension. In Proceedings of CoNLL, pages 333–342.
  22. Dividemix: Learning with noisy labels as semi-supervised learning. In Proceedings of ICLR.
  23. Global constraints with prompting for zero-shot event argument classification. In Findings of EACL, pages 2482–2493.
  24. Event extraction as machine reading comprehension. In Proceedings of EMNLP, pages 1641–1651.
  25. Translation-based implicit annotation projection for zero-shot cross-lingual event argument extraction. In Proceedings of SIGIR, pages 2076–2081.
  26. Event extraction as question generation and answering. In Proceedings of ACL, pages 1666–1688.
  27. Summarization as indirect supervision for relation extraction. arXiv preprint arXiv:2205.09837.
  28. Label-noise learning with intrinsically long-tailed data. In Proceedings of ICCV, pages 1369–1378.
  29. Zero-shot event extraction via transfer learning: Challenges and insights. In Proceedings of ACL, pages 322–332.
  30. Shengfei Lyu and Huanhuan Chen. 2021. Relation classification with entity type restriction. In Proceedings of ACL, pages 390–395.
  31. Yueming Lyu and Ivor W Tsang. 2019. Curriculum loss: Robust learning and generalization against label corruption. In Proceedings of ICLR.
  32. Wider & closer: Mixture of short-channel distillers for zero-shot cross-lingual named entity recognition. arXiv preprint arXiv:2212.03506.
  33. Shine: Syntax-augmented hierarchical interactive encoder for zero-shot cross-lingual information extraction. arXiv preprint arXiv:2305.12389.
  34. Normalized loss functions for deep learning with noisy labels. In Proceedings of ICML, volume 119, pages 6543–6553.
  35. Eran Malach and Shai Shalev-Shwartz. 2017. Decoupling" when to update" from" how to update". In Proceedings of NeurIPS, pages 960–970.
  36. Improving zero-shot event extraction via sentence simplification. In Workshop of EMNLP, pages 32–43.
  37. Can gradient clipping mitigate label noise? In Proceedings of ICLR.
  38. Abiola Obamuyide and Andreas Vlachos. 2018. Zero-shot relation classification as textual entailment. In Proceedings of the FEVER, pages 72–78.
  39. Sinno Jialin Pan and Qiang Yang. 2009. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, pages 1345–1359.
  40. Mahdi Rahimi and Mihai Surdeanu. 2023. Improving zero-shot relation classification via automatically-acquired entailment templates. In Proceedings of Workshop on RepL4NLP, pages 187–195.
  41. Training deep neural networks on noisy labels with bootstrapping. In Proceedings of ICLR Workshop.
  42. Label verbalization and entailment for effective zero-and few-shot relation extraction. In Proceedings of EMNLP, pages 1199–1212.
  43. Textual entailment for event argument extraction: Zero- and few-shot with multi-source learning. In Findings of NAACL-HLT, pages 2439–2455.
  44. Zs4ie: A toolkit for zero-shot information extraction with simple verbalizations. In Proceedings of NAACL-HLT, pages 27–38.
  45. Parallel instance query network for named entity recognition. In Proceedings of ACL, pages 947–961.
  46. Meta-weight-net: Learning an explicit mapping for sample weighting. In Proceedings of NeurIPS, pages 1917–1928.
  47. Combating label noise in deep learning using abstention. arXiv preprint arXiv:1905.10964.
  48. Revisiting unsupervised relation extraction. In Proceedings of ACL, pages 7498–7505.
  49. One-shot to weakly-supervised relation classification using language models. In Proceedings of AKBC.
  50. Symmetric cross entropy for robust learning with noisy labels. In Proceedings of ICCV, pages 322–330.
  51. Single-/multi-source cross-lingual ner via teacher-student learning on unlabeled data in target language. In Proceedings of ACL, pages 6505–6514.
  52. Exploring pre-trained language models for event extraction and generation. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 5284–5294.
  53. Searching to exploit memorization effect in learning from corrupted labels. arXiv preprint arXiv:1911.02377.
  54. How does disagreement help generalization against label corruption? In Proceedings of ICML, pages 7164–7173.
  55. Understanding deep learning requires rethinking generalization. In Proceedings of ICLR.
  56. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3):107–115.
  57. Zero-shot label-aware event trigger and argument classification. In Findings of ACL-IJCNLP, pages 1331–1340.
  58. Aligning instruction tasks unlocks large language models as zero-shot relation extractors. In ACL, pages 794–812.
  59. Zhilu Zhang and Mert Sabuncu. 2018. Generalized cross entropy loss for training deep neural networks with noisy labels. In Proceedings of NeurIPS, pages 8792–8802.
  60. Re-matching: A fine-grained semantic matching method for zero-shot relation extraction. In Proceedings of ACL, pages 6680–6691.
  61. Zexuan Zhong and Danqi Chen. 2021. A frustratingly easy approach for entity and relation extraction. In Proceedings of NAACL, pages 50–61.
  62. Wenxuan Zhou and Muhao Chen. 2021. Learning from noisy labels for entity-centric information extraction. In Proceedings of EMNLP, pages 5381–5392.
  63. Enwei Zhu and Jinpeng Li. 2022. Boundary smoothing for named entity recognition. In Proceedings of ACL, pages 7096–7108.
  64. A joint neural model for information extraction with global features. PID https://aclanthology.org/2020.acl-main.713/.
  65. Multilingual Entity and Relation Extraction Dataset and Model. PID https://aclanthology.org/2021.eacl-main.166.
  66. Hongmin Xiao. 2022. Wiki80. PID https://figshare.com/articles/dataset/Wiki80/19323371.
  67. Victor Zhong and Yuhao Zhang and Danqi Chen and Gabor Angeli and Christopher Manning. 2018. TAC Relation Extraction Dataset. ISLRN 927-859-759-915-2. PID https://catalog.ldc.upenn.edu/LDC2018T24.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up