Dual Learning for Machine Translation (1611.00179v1)

Abstract: While neural machine translation (NMT) is making good progress in the past two years, tens of millions of bilingual sentence pairs are needed for its training. However, human labeling is very costly. To tackle this training data bottleneck, we develop a dual-learning mechanism, which can enable an NMT system to automatically learn from unlabeled data through a dual-learning game. This mechanism is inspired by the following observation: any machine translation task has a dual task, e.g., English-to-French translation (primal) versus French-to-English translation (dual); the primal and dual tasks can form a closed loop, and generate informative feedback signals to train the translation models, even if without the involvement of a human labeler. In the dual-learning mechanism, we use one agent to represent the model for the primal task and the other agent to represent the model for the dual task, then ask them to teach each other through a reinforcement learning process. Based on the feedback signals generated during this process (e.g., the language-model likelihood of the output of a model, and the reconstruction error of the original sentence after the primal and dual translations), we can iteratively update the two models until convergence (e.g., using the policy gradient methods). We call the corresponding approach to neural machine translation \emph{dual-NMT}. Experiments show that dual-NMT works very well on English$\leftrightarrow$French translation; especially, by learning from monolingual data (with 10% bilingual data for warm start), it achieves a comparable accuracy to NMT trained from the full bilingual data for the French-to-English translation task.

• The paper introduces a dual learning mechanism that leverages monolingual data with reinforcement learning to mitigate parallel data scarcity in NMT.
• The method employs a dual-agent game where forward and backward translation models iteratively refine translations using language model and reconstruction rewards.
• The approach achieves significant BLEU improvements and comparable performance using only 10% of parallel data in low-resource settings.

Dual Learning for Machine Translation

The paper "Dual Learning for Machine Translation" by Yingce Xia et al. introduces an innovative approach to mitigating the data scarcity problem in Neural Machine Translation (NMT). The primary contribution of this work is the dual-learning mechanism, a novel methodology that enables NMT systems to leverage monolingual data effectively through a reinforcement learning framework.

Background and Motivation

Conventional machine translation systems, including both phrase-based statistical methods and recent neural approaches, significantly depend on large aligned parallel corpora. However, the acquisition of such parallel datasets is cost-intensive and often limited, curtailing the efficacy of translation models. While there has been some progress in utilizing monolingual data, current methods either rely solely on improving LLMs or generate pseudo bilingual sentence pairs without ensuring their quality.

Proposed Method: Dual Learning Mechanism

The core idea of dual learning is to exploit the intrinsic duality in translation tasks—such as English-to-French (primal) and French-to-English (dual)—to create a closed feedback loop. This loop allows two translation models to iteratively improve each other by generating and evaluating translations reciprocally.

Mechanism

1. Dual-Agent Game: Two agents, A and B, represent the primal and dual translation tasks, respectively. Agent A translates a sentence from English to French, and agent B translates it back to English. The quality of these translations is then assessed using LLMs for each language and reconstruction accuracy.
2. Feedback and Update:
   • LLM Reward: The naturalness of the translation is evaluated using pre-trained LLMs specific to the target language.
   • Reconstruction Reward: The reconstruction error, i.e., the discrepancy between the original sentence and the back-translated sentence, provides a measure of translation fidelity.
3. Reinforcement Learning: The translation models are iteratively updated using policy gradient methods to maximize the combined rewards from the LLM and reconstruction feedback.

Results and Discussion

The dual-learning approach was empirically validated on the English-French translation task using two bilingual data settings: one with the full WMT'14 dataset (12M sentence pairs) and a reduced dataset (10% of the full data). The results indicate that:

• Performance: Dual-NMT significantly outperforms both the standard NMT and pseudo-NMT methods, achieving a 2.1/3.4 BLEU score improvement in English-to-French translation and a 2.3/5.2 point improvement in the opposite direction for the large/small data settings respectively.
• Data Efficiency: Notable is the dual-NMT's ability to achieve comparable performance using only 10% of the parallel data, underscoring its efficiency in data-starved scenarios.
• Reconstruction Accuracy: The self-reconstruction performance, measured by the BLEU score after translating a sentence forward and backward, demonstrated substantial improvement, indicating enhanced translation fidelity.

Implications and Future Work

The dual-learning mechanism opens multiple avenues for improving and expanding machine translation systems:

1. General Applicability: The concept of dual learning can be extended beyond machine translation to any dual-task scenario, such as speech recognition and text-to-speech, image captioning and generation, or question answering and generation.
2. Closed-Loop Learning: The technique can be scaled to involve more than two languages or tasks, forming closed loops where multiple translation models mutually refine each other.
3. Learning from Scratch: A promising direction is the potential to train models purely from monolingual data and a lexical dictionary, eliminating the dependency on parallel corpora entirely.

The reinforcement learning framework presented by dual-NMT not only alleviates the data constraint in NMT but also exemplifies how deep reinforcement learning can be utilized for real-world NLP tasks beyond game playing scenarios. Future research could explore its application in broader contexts and enhance its robustness and efficiency further.

In summary, this paper provides a significant advancement in machine translation by leveraging monolingual data effectively, thus opening new possibilities for NLP tasks where parallel data is scarce or unavailable.