Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

mCL-NER: Cross-Lingual Named Entity Recognition via Multi-view Contrastive Learning (2308.09073v2)

Published 17 Aug 2023 in cs.CL

Abstract: Cross-lingual named entity recognition (CrossNER) faces challenges stemming from uneven performance due to the scarcity of multilingual corpora, especially for non-English data. While prior efforts mainly focus on data-driven transfer methods, a significant aspect that has not been fully explored is aligning both semantic and token-level representations across diverse languages. In this paper, we propose Multi-view Contrastive Learning for Cross-lingual Named Entity Recognition (mCL-NER). Specifically, we reframe the CrossNER task into a problem of recognizing relationships between pairs of tokens. This approach taps into the inherent contextual nuances of token-to-token connections within entities, allowing us to align representations across different languages. A multi-view contrastive learning framework is introduced to encompass semantic contrasts between source, codeswitched, and target sentences, as well as contrasts among token-to-token relations. By enforcing agreement within both semantic and relational spaces, we minimize the gap between source sentences and their counterparts of both codeswitched and target sentences. This alignment extends to the relationships between diverse tokens, enhancing the projection of entities across languages. We further augment CrossNER by combining self-training with labeled source data and unlabeled target data. Our experiments on the XTREME benchmark, spanning 40 languages, demonstrate the superiority of mCL-NER over prior data-driven and model-based approaches. It achieves a substantial increase of nearly +2.0 $F_1$ scores across a broad spectrum and establishes itself as the new state-of-the-art performer.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Zero-Resource Cross-Lingual Named Entity Recognition. In AAAI 2020, 7415–7423.
  2. xCoT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning. arXiv preprint arXiv:2401.07037.
  3. Dictbert: Dictionary description knowledge enhanced language model pre-training via contrastive learning. arXiv preprint arXiv:2208.00635.
  4. A Simple Framework for Contrastive Learning of Visual Representations. CoRR, abs/2002.05709.
  5. AdvPicker: Effectively Leveraging Unlabeled Data via Adversarial Discriminator for Cross-Lingual NER. CoRR, abs/2106.02300.
  6. Debiased contrastive learning. NIPS 2020, 33: 8765–8775.
  7. Unsupervised Cross-lingual Representation Learning at Scale. CoRR, abs/1911.02116.
  8. Container: Few-shot named entity recognition via contrastive learning. arXiv preprint arXiv:2109.07589.
  9. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. CoRR, abs/1810.04805.
  10. Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821.
  11. Declutr: Deep contrastive learning for unsupervised textual representations. arXiv preprint arXiv:2006.03659.
  12. LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translation. In EMNLP 2022, 2862–2872.
  13. Cross-Lingual Contrastive Learning for Fine-Grained Entity Typing for Low-Resource Languages. In AAAI 2022, 2241–2250.
  14. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9729–9738.
  15. Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network. arXiv:2006.05702.
  16. XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization. arXiv:2003.11080.
  17. Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NER. CoRR, abs/1909.00153.
  18. Supervised contrastive learning. Advances in neural information processing systems, 33: 18661–18673.
  19. Unified Named Entity Recognition as Word-Word Relation Classification. CoRR, abs/2112.10070.
  20. Unsupervised Cross-lingual Adaptation for Sequence Tagging and Beyond. arXiv:2010.12405.
  21. MulDA: A Multilingual Data Augmentation Framework for Low-Resource Cross-Lingual NER. In ACL 2021, 5834–5846.
  22. Decoupled Weight Decay Regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019.
  23. XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders. CoRR, abs/2012.15547.
  24. Cheap translation for cross-lingual named entity recognition. In EMNLP 2017, 2536–2545.
  25. Multi-Task Transformer with Relation-Attention and Type-Attention for Named Entity Recognition. In ICASSP 2023, 1–5.
  26. Weakly Supervised Cross-Lingual Named Entity Recognition via Effective Annotation and Representation Projection. CoRR, abs/1707.02483.
  27. Balanced MSE for Imbalanced Visual Regression. CoRR, abs/2203.16427.
  28. Sang, E. F. T. K. 2002. Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition. CoRR, cs.CL/0209010.
  29. Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. CoRR, cs.CL/0306050.
  30. OneRel: Joint Entity and Relation Extraction with One Module in One Step. CoRR, abs/2203.05412.
  31. Offline bilingual word vectors, orthogonal transformations and the inverted softmax. CoRR, abs/1702.03859.
  32. UniRel: Unified Representation and Interaction for Joint Relational Triple Extraction. CoRR, abs/2211.09039.
  33. Visualizing Data using t-SNE. Journal of Machine Learning Research, 9(86): 2579–2605.
  34. Harnessing Pre-Trained Neural Networks with Rules for Formality Style Transfer. In EMNLP 2019, 3571–3576.
  35. Formality Style Transfer with Shared Latent Space. In COLING 2020, 2236–2249.
  36. MT4CrossOIE: Multi-stage Tuning for Cross-lingual Open Information Extraction. arXiv preprint arXiv:2308.06552.
  37. Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language. In ACL 2020, 6505–6514.
  38. UniTrans : Unifying Model Transfer and Data Transfer for Cross-Lingual Named Entity Recognition with Unlabeled Data. In IJCAI 2020, 3926–3932.
  39. Enhanced Meta-Learning for Cross-Lingual Named Entity Recognition with Minimal Resources. In AAAI 2020, 9274–9281.
  40. Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT. CoRR, abs/1904.09077.
  41. A Dataset for Low-Resource Stylized Sequence-to-Sequence Generation. In AAAI 2020, 9290–9297.
  42. Neural Cross-Lingual Named Entity Recognition with Minimal Resources. CoRR, abs/1808.09861.
  43. CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation. In Findings of EMNLP 2022, 486–496.
  44. Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task. CoRR, abs/2111.02086.
  45. Improving Neural Machine Translation with Soft Template Prediction. In ACL 2020, 5979–5989.
  46. Alternating Language Modeling for Cross-Lingual Pre-Training. In AAAI 2020, 9386–9393.
  47. Multilingual Agreement for Multilingual Neural Machine Translation. In ACL 2021, 233–239.
  48. High-resource Language-specific Training for Multilingual Neural Machine Translation. In IJCAI 2022, 4461–4467. ijcai.org.
  49. UM4: Unified Multilingual Multiple Teacher-Student Model for Zero-Resource Neural Machine Translation. In IJCAI 2022, 4454–4460.
  50. Packed Levitated Marker for Entity and Relation Extraction. CoRR, abs/2109.06067.
  51. Semi-Open Information Extraction. In WWW 2021, 1661–1672. ACM / IW3C2.
  52. Named Entity Recognition as Dependency Parsing. CoRR, abs/2005.07150.
  53. Improving self-training for cross-lingual named entity recognition with contrastive and prototype learning. arXiv preprint arXiv:2305.13628.
  54. ConNER: Consistency Training for Cross-lingual Named Entity Recognition. In EMNLP 2022, 8438–8449.
  55. Boundary Smoothing for Named Entity Recognition. CoRR, abs/2204.12031.
Citations (17)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now