X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale (2410.03115v1)

Abstract: LLMs have achieved remarkable success across various NLP tasks, yet their focus has predominantly been on English due to English-centric pre-training and limited multilingual data. While some multilingual LLMs claim to support for hundreds of languages, models often fail to provide high-quality response for mid- and low-resource languages, leading to imbalanced performance heavily skewed in favor of high-resource languages like English and Chinese. In this paper, we prioritize quality over scaling number of languages, with a focus on multilingual machine translation task, and introduce X-ALMA, a model designed with a commitment to ensuring top-tier performance across 50 diverse languages, regardless of their resource levels. X-ALMA surpasses state-of-the-art open-source multilingual LLMs, such as Aya-101 and Aya-23, in every single translation direction on the FLORES and WMT'23 test datasets according to COMET-22. This is achieved by plug-and-play language-specific module architecture to prevent language conflicts during training and a carefully designed training regimen with novel optimization methods to maximize the translation performance. At the final stage of training regimen, our proposed Adaptive Rejection Preference Optimization (ARPO) surpasses existing preference optimization methods in translation tasks.

Overview of X-ALMA: Enhancing Multilingual Translation with Plug-and-Play Architecture and Adaptive Optimization

The paper "X-ALMA: Plug-and-Play Modules and Adaptive Rejection for Quality Translation at Scale" presents a novel approach to multilingual machine translation by addressing the limitations inherent in current LLMs. The authors introduce X-ALMA, a model that prioritizes translation quality across 50 languages, transcending the typical focus on high-resource languages.

Key Contributions

X-ALMA's main innovations revolve around two core concepts: a plug-and-play architectural framework and a sophisticated training regimen inclusive of Adaptive-Rejection Preference Optimization (ARPO).

Architecture

The model employs a plug-and-play architecture, structuring language-specific (LS) modules around a dense base model inspired by LLaMA-2. These modules are organized into eight language groups to reduce training conflicts and are engaged based on input language characteristics. This modular design offers adaptability, allowing for three deployment strategies:

1. Single Module Loading: Activating only the necessary LS module saves memory resources.
2. Merged Module Deployment: All LS modules are combined into a single model, maintaining parameter efficiency.
3. Comprehensive MoE Integration: All modules can be simultaneously loaded in a manner akin to the Mixture-of-Experts (MoE) architecture.

Training Recipe

The five-stage training process integrates both pre-training and post-training strategies:

1. Monolingual Fine-Tuning: Initial adaptation to diverse languages.
2. Language-Specific Module Training: Enhancing module specialization.
3. Pseudo-Monolingual Training: Facilitating multilingual alignment.
4. Supervised Fine-Tuning (SFT): Utilizing high-quality parallel datasets.
5. Adaptive-Rejection Preference Optimization (ARPO): Refining translation outputs by mitigating the over-rejection phenomenon found in preference learning.

Evaluations and Results

X-ALMA sets a new benchmark by outperforming state-of-the-art models like Aya-101 and Aya-23 on both the FLORES-200 and WMT'23 datasets. Metrics used include COMET-22 and XCOMET-XL. The model also mitigates the 'curse of multilinguality', exemplifying robust performance regardless of language resource levels.

Implications and Future Directions

This research extends beyond improving translation quality to suggest broader applicability in multilingual NLP tasks. The modular design and adaptive optimization techniques could influence future LLM development, particularly in scaling models while preserving language-specific nuances.

The introduction of ARPO suggests a new pathway for preference optimization, addressing the balance between translation accuracy and stylistic fidelity. Future work may focus on enhancing adaptive methods to further optimize multilingual alignments and performance across diverse linguistic contexts.

Overall, X-ALMA represents a significant step forward in multilingual machine translation, balancing scalability with quality, and offering a framework adaptable to future advancements in natural language processing.