Bilex Rx: Lexical Data Augmentation for Massively Multilingual Machine Translation (2303.15265v1)

Abstract: Neural machine translation (NMT) has progressed rapidly over the past several years, and modern models are able to achieve relatively high quality using only monolingual text data, an approach dubbed Unsupervised Machine Translation (UNMT). However, these models still struggle in a variety of ways, including aspects of translation that for a human are the easiest - for instance, correctly translating common nouns. This work explores a cheap and abundant resource to combat this problem: bilingual lexica. We test the efficacy of bilingual lexica in a real-world set-up, on 200-language translation models trained on web-crawled text. We present several findings: (1) using lexical data augmentation, we demonstrate sizable performance gains for unsupervised translation; (2) we compare several families of data augmentation, demonstrating that they yield similar improvements, and can be combined for even greater improvements; (3) we demonstrate the importance of carefully curated lexica over larger, noisier ones, especially with larger models; and (4) we compare the efficacy of multilingual lexicon data versus human-translated parallel data. Finally, we open-source GATITOS (available at https://github.com/google-research/url-nlp/tree/main/gatitos), a new multilingual lexicon for 26 low-resource languages, which had the highest performance among lexica in our experiments.

• The paper demonstrates that bilingual lexical augmentation methods significantly improve translation quality, particularly for low-resource and unsupervised languages.
• The study employs three strategies—codeswitching, GLOWUP, and raw token pairs—evaluating their impact on ChrF scores using datasets like Flores-200 and Gatones.
• The research finds that high-quality resources like Gatitos outperform larger, noisier lexica, underscoring the importance of lexical quality in machine translation.

Analysis of Lexical Data Augmentation in Multilingual Machine Translation

The paper presents an empirical investigation into the application of lexical data augmentation methods to improve multilingual machine translation (MT), particularly focusing on low-resource and unsupervised languages. By harnessing bilingual lexica, the researchers aim to enhance cross-lingual vocabulary alignment, addressing a common pitfall in unsupervised MT systems—mistranslation of semantically similar words, a problem notably prevalent among nouns like animal names.

Methodological Framework

The research evaluates three primary augmentation strategies using the Panlex database and a newly introduced resource, Gatitos. These strategies, employed during training, include:

1. Codeswitching: This involves substituting source text words with their dictionary translations.
2. Guiding Lexical Output with Understandable Prompts (GLOWUP): This method appends lexicon translations to source sentences to provide hints about potential translations.
3. Raw Token Pairs: Used as parallel data, leveraging direct lexicon entries to guide translation models.

The experiments involve a baseline model deploying monolingual and parallel data without augmentations and compare it with models implementing each augmentation method. Performance is primarily assessed using ChrF scores on the Flores-200 and Gatones evaluation sets.

Empirical Findings

The key findings demonstrate that augmentations involving monolingual data lead to statistically significant improvements in translation quality for unsupervised and low-resource languages. Notably, the CodeswitchMonoGatiPanlex model, which incorporates both codeswitching and raw token pairs, shows the highest average gain in ChrF scores, particularly in unsupervised settings.

Additionally, the efficacy of augmentations is highlighted further with the deployment of the Gatitos dataset, which provides curated lexical entries, yielding a more significant performance increase than the larger, noisier Panlex dataset alone. This underscores the value of lexicon quality over quantity in training.

Implications and Future Directions

The paper accentuates the practical implications of using bilingual lexica in MT systems, especially where resource constraints limit large-scale parallel data acquisition. The methods outlined could considerably improve low-resource language support by providing cost-effective and scalable solutions.

Future research directions could explore:

• Dynamic augmentation methods that adapt during training to continually enhance vocabulary alignment.
• Integration with emerging LLMs, potentially improving performance in even higher-resource settings.
• Quality-centric lexicon curation, building on the promising results of the Gatitos data, to refine augmentation strategies further.

By contributing to the corpus of knowledge on bilingual lexica in MT, this research paves the way for more nuanced approaches in reducing common translation errors and refining output quality across diverse languages, reinforcing MT systems' robustness and efficacy in multilingual contexts.