Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Going Beyond Word Matching: Syntax Improves In-context Example Selection for Machine Translation (2403.19285v2)

Published 28 Mar 2024 in cs.CL

Abstract: In-context learning (ICL) is the trending prompting strategy in the era of LLMs, where a few examples are demonstrated to evoke LLMs' power for a given task. How to select informative examples remains an open issue. Previous works on in-context example selection for machine translation (MT) focus on superficial word-level features while ignoring deep syntax-level knowledge. In this paper, we propose a syntax-based in-context example selection method for MT, by computing the syntactic similarity between dependency trees using Polynomial Distance. In addition, we propose an ensemble strategy combining examples selected by both word-level and syntax-level criteria. Experimental results between English and 6 common languages indicate that syntax can effectively enhancing ICL for MT, obtaining the highest COMET scores on 11 out of 12 translation directions.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (26)
  1. In-context examples selection for machine translation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8857–8873, Toronto, Canada. Association for Computational Linguistics.
  2. ParaCrawl: Web-scale acquisition of parallel corpora. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4555–4567, Online. Association for Computational Linguistics.
  3. Dorian Brown. 2020. Rank-BM25: A Collection of BM25 Algorithms in Python.
  4. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc.
  5. Michael Collins and Nigel Duffy. 2002. New ranking algorithms for parsing and tagging: Kernels over discrete structures, and the voted perceptron. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 263–270, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
  6. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672.
  7. Susan Craw. 2017. Manhattan distance. Encyclopedia of Machine Learning and Data Mining, pages 790–791.
  8. Automatic web news extraction using tree edit distance. In The Web Conference.
  9. A survey for in-context learning. arXiv preprint arXiv:2301.00234.
  10. spaCy: Industrial-strength Natural Language Processing in Python.
  11. To ship or not to ship: An extensive evaluation of automatic metrics for machine translation. In Proceedings of the Sixth Conference on Machine Translation, pages 478–494, Online. Association for Computational Linguistics.
  12. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of Machine Translation Summit X: Papers, pages 79–86, Phuket, Thailand.
  13. Few-shot learning with multilingual generative language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9019–9052, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  14. Quantifying syntax similarity with a polynomial representation of dependency trees. arXiv preprint arXiv:2211.07005.
  15. CTQScorer: Combining multiple features for in-context example selection for machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7736–7752, Singapore. Association for Computational Linguistics.
  16. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837.
  17. Alessandro Moschitti. 2006. Efficient convolution kernels for dependency and constituent syntactic trees. In Machine Learning: ECML 2006, pages 318–329, Berlin, Heidelberg. Springer Berlin Heidelberg.
  18. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186–191, Brussels, Belgium. Association for Computational Linguistics.
  19. Unbabel’s participation in the WMT20 metrics shared task. In Proceedings of the Fifth Conference on Machine Translation, pages 911–920, Online. Association for Computational Linguistics.
  20. Okapi at trec-3. In Text Retrieval Conference.
  21. WikiMatrix: Mining 135M parallel sentences in 1620 language pairs from Wikipedia. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1351–1361, Online. Association for Computational Linguistics.
  22. Fast kernels for string and tree matching. Kernel methods in computational biology, 15(113-130):1.
  23. Transformers learn in-context by gradient descent. In International Conference on Machine Learning, pages 35151–35174. PMLR.
  24. Philip Williams and Philipp Koehn. 2014. Syntax-based statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts, Doha, Qatar. Association for Computational Linguistics.
  25. Improved neural machine translation with source syntax. In International Joint Conference on Artificial Intelligence.
  26. Prompting large language model for machine translation: A case study. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 41092–41110. PMLR.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Chenming Tang (6 papers)
  2. Zhixiang Wang (30 papers)
  3. Yunfang Wu (50 papers)