Papers
Topics
Authors
Recent
Search
2000 character limit reached

Retrieving Examples from Memory for Retrieval Augmented Neural Machine Translation: A Systematic Comparison

Published 3 Apr 2024 in cs.CL | (2404.02835v1)

Abstract: Retrieval-Augmented Neural Machine Translation (RAMT) architectures retrieve examples from memory to guide the generation process. While most works in this trend explore new ways to exploit the retrieved examples, the upstream retrieval step is mostly unexplored. In this paper, we study the effect of varying retrieval methods for several translation architectures, to better understand the interplay between these two processes. We conduct experiments in two language pairs in a multi-domain setting and consider several downstream architectures based on a standard autoregressive model, an edit-based model, and a LLM with in-context learning. Our experiments show that the choice of the retrieval technique impacts the translation scores, with variance across architectures. We also discuss the effects of increasing the number and diversity of examples, which are mostly positive across the board.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. In-context examples selection for machine translation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8857–8873, Toronto, Canada. Association for Computational Linguistics.
  2. Roee Aharoni and Yoav Goldberg. 2020. Unsupervised domain clusters in pretrained language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7747–7763, Online. Association for Computational Linguistics.
  3. The falcon series of open language models.
  4. Tower: An open multilingual large language model for translation-related tasks.
  5. Rachel Bawden and François Yvon. 2023. Investigating the translation performance of a large multilingual language model: the case of BLOOM. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pages 157–170, Tampere, Finland. European Association for Machine Translation.
  6. BLOOM: a 176b-parameter open-access multilingual language model.
  7. Towards example-based NMT with multi-Levenshtein transformers. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1830–1846, Singapore. Association for Computational Linguistics.
  8. Lynne Bowker. 2002. Computer-aided translation technology: A practical introduction. University of Ottawa Press.
  9. Bram Bulte and Arda Tezcan. 2019. Neural fuzzy repair: Integrating fuzzy matches into neural machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1800–1809, Florence, Italy. Association for Computational Linguistics.
  10. Neural machine translation with monolingual translation memory. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7307–7318, Online. Association for Computational Linguistics.
  11. Recent advances in example-based machine translation. Computational Linguistics, 30:516–520.
  12. Neural machine translation with contrastive translation memories. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3591–3601, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  13. Multi-domain neural machine translation through unsupervised adaptation. In Proceedings of the Second Conference on Machine Translation, pages 127–137, Copenhagen, Denmark. Association for Computational Linguistics.
  14. Jade Goldstein and Jaime Carbonell. 1998. Summarization: (1) using MMR for diversity- based reranking and (2) evaluating summaries. In TIPSTER TEXT PROGRAM PHASE III: Proceedings of a Workshop held at Baltimore, Maryland, October 13-15, 1998, pages 181–195, Baltimore, Maryland, USA. Association for Computational Linguistics.
  15. Levenshtein transformer. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
  16. Search Engine Guided Neural Machine Translation. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1).
  17. Coverage-based example selection for in-context learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13924–13950, Singapore. Association for Computational Linguistics.
  18. Efficient nearest neighbor language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5703–5714, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  19. Fast and accurate neural machine translation with translation memory. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3170–3180, Online. Association for Computational Linguistics.
  20. How good are GPT models at machine translation? a comprehensive evaluation. CoRR, abs/2302.09210.
  21. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  22. Nearest Neighbor Machine Translation. In Proceedings of the International Conference on Learning Representations.
  23. Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation, pages 28–39, Vancouver. Association for Computational Linguistics.
  24. Philipp Koehn and Jean Senellart. 2010. Convergence of translation memory and statistical machine translation. In Proceedings of the Second Joint EM+/CNGL Workshop: Bringing MT to the User: Research on Integrating MT in the Translation Industry, pages 21–32, Denver, Colorado, USA. Association for Machine Translation in the Americas.
  25. The BigScience ROOTS Corpus: A 1.6 TB Composite Multilingual Dataset. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
  26. A survey on retrieval-augmented text generation. CoRR, abs/2202.01110.
  27. A survey on retrieval-augmented text generation.
  28. CTQScorer: Combining multiple features for in-context example selection for machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7736–7752, Singapore. Association for Computational Linguistics.
  29. Chunk-based nearest neighbor machine translation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4228–4245, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  30. Fast nearest neighbor machine translation. In Findings of the Association for Computational Linguistics: ACL 2022, pages 555–565, Dublin, Ireland. Association for Computational Linguistics.
  31. Adaptive machine translation with large language models. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pages 227–237, Tampere, Finland. European Association for Machine Translation.
  32. Augmenting large language model translators via translation memories. In Findings of the Association for Computational Linguistics: ACL 2023, pages 10287–10299, Toronto, Canada. Association for Computational Linguistics.
  33. Makoto Nagao. 1984. A framework of a mechanical translation between Japanese and English by analogy principle. In Artificial and human intelligence. Elsevier Science Publishers. B.V.
  34. Nearest neighbor non-autoregressive text generation. CoRR, abs/2208.12496.
  35. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
  36. Priming neural machine translation. In Proceedings of the Fifth Conference on Machine Translation, pages 516–527, Online. Association for Computational Linguistics.
  37. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186–191, Brussels, Belgium. Association for Computational Linguistics.
  38. Prompting neural machine translation with translation memories. Proceedings of the AAAI Conference on Artificial Intelligence, 37(11):13519–13527.
  39. COMET-22: Unbabel-IST 2022 submission for the metrics shared task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 578–585, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  40. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends® in Information Retrieval, 3(4):333–389.
  41. Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5):206–215.
  42. Suzanna Sia and Kevin Duh. 2023a. In-context learning as maintaining coherency: A study of on-the-fly machine translation using large language models. In Proceedings of Machine Translation Summit XIX, Vol. 1: Research Track, pages 173–185, Macau SAR, China. Asia-Pacific Association for Machine Translation.
  43. Suzanna Sia and Kevin Duh. 2023b. In-context learning as maintaining coherency: A study of on-the-fly machine translation using large language models.
  44. Harold Somers. 1999. Review article: Example-based machine translation. Machine Translation, 14(2):113–157.
  45. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in OPUS. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 2214–2218, Istanbul, Turkey. European Language Resources Association (ELRA).
  46. Llama: Open and efficient foundation language models.
  47. Prompting PaLM for translation: Assessing strategies and performance. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15406–15427, Toronto, Canada. Association for Computational Linguistics.
  48. Graph based translation memory for neural machine translation. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):7297–7304.
  49. Boosting neural machine translation with similar translations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1580–1590, Online. Association for Computational Linguistics.
  50. Boosting neural machine translation with similar translations. In Proceedings of the 15th Biennial Conference of the Association for Machine Translation in the Americas (Volume 2: Users and Providers Track and Government Track), pages 282–292, Orlando, USA. Association for Machine Translation in the Americas.
  51. Integrating translation memories into non-autoregressive machine translation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1326–1338, Dubrovnik, Croatia. Association for Computational Linguistics.
  52. Prompting large language model for machine translation: A case study. In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org.
  53. Guiding neural machine translation with retrieved translation pieces. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1325–1335, New Orleans, Louisiana. Association for Computational Linguistics.
  54. Towards a unified training for Levenshtein transformer. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5.
  55. Adaptive nearest neighbor machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 368–374, Online. Association for Computational Linguistics.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 3 tweets with 9 likes about this paper.

Sign Up for Free