Fully Character-Level Neural Machine Translation without Explicit Segmentation

Abstract: Most existing machine translation systems operate at the level of words, relying on explicit segmentation to extract tokens. We introduce a neural machine translation (NMT) model that maps a source character sequence to a target character sequence without any segmentation. We employ a character-level convolutional network with max-pooling at the encoder to reduce the length of source representation, allowing the model to be trained at a speed comparable to subword-level models while capturing local regularities. Our character-to-character model outperforms a recently proposed baseline with a subword-level encoder on WMT'15 DE-EN and CS-EN, and gives comparable performance on FI-EN and RU-EN. We then demonstrate that it is possible to share a single character-level encoder across multiple languages by training a model on a many-to-one translation task. In this multilingual setting, the character-level encoder significantly outperforms the subword-level encoder on all the language pairs. We observe that on CS-EN, FI-EN and RU-EN, the quality of the multilingual character-level translation even surpasses the models specifically trained on that language pair alone, both in terms of BLEU score and human judgment.

• The paper introduces a fully character-level NMT model that processes text without explicit segmentation, challenging conventional subword-based methods.
• It employs a convolutional encoder with max-pooling and bidirectional GRUs to capture both local and long-range dependencies efficiently.
• The approach demonstrates enhanced performance in bilingual and multilingual settings while reducing vocabulary dependency and enabling effective parameter sharing.

Fully Character-Level Neural Machine Translation without Explicit Segmentation

The paper "Fully Character-Level Neural Machine Translation without Explicit Segmentation" presents an in-depth study on neural machine translation (NMT) models that operate at the character level, eschewing traditional segmentation techniques. This research is a significant contribution to the field, examining both the benefits and challenges of character-level models in bilingual and multilingual contexts.

Methodology and Architecture

The researchers propose a model architecture that processes source input as sequences of individual characters, mapping them directly to target languages. This is achieved through a character-level convolutional network equipped with max-pooling at the encoder to reduce input sequence length while effectively capturing local regularities. The model incorporates a stack of convolutional and highway layers before engaging a bidirectional GRU to handle long-range dependencies.

The encoder begins by mapping each source character into embeddings, and a series of filters process these embeddings to capture n-gram patterns. Max-pooling reduces sequence lengths, making training computationally feasible. The study posits two architecture configurations: a bilingual model specific to a language pair and a multilingual model that translates from multiple languages to a single target language.

Key Findings

1. Performance Metrics: Character-to-character models demonstrated superior performance compared to subword-level baselines in bilingual settings, particularly for DE-EN and CS-EN language pairs. They proved comparable on FI-EN and RU-EN tasks.
2. Multilingual Efficiency: In multilingual settings, character-level models significantly outperformed subword-level counterparts for all tested language pairs. The character-level approach resulted in increased parameter efficiency, enabling the model to share capacity across languages effectively.
3. Robustness: Character-level models handled various linguistic phenomena such as rare words, morphological variations, and intra-sentence code-switching more robustly than subword models.

Implications

Character-level translation models offer several practical benefits: they eliminate the need for predefined token vocabularies, are less susceptible to word segmentation issues, and are inherently open-vocabulary. These advantages make them particularly suitable for languages with rich morphology and in multilingual contexts where overlapping alphabets exist. The scalability of these models to many languages without enlarging the model size is a notable achievement.

Future Directions

The promising results of this study suggest that future work could explore extending the multilingual model to handle multiple target languages, potentially developing many-to-many translation systems. Further investigation into optimizing model architectures and hyperparameters could yield efficiency gains, making character-level translation even more viable for real-world applications.

This research substantiates the viability of fully character-level models in both bilingual and multilingual NMT, indicating a meaningful direction for future advancements in translation technology. The paper encourages the field to reconsider the fundamental units of translation, highlighting the utility of character-level approaches in achieving flexible and scalable translation systems.