Terminology-Constrained MT

• TC-MT is a framework that enforces pre-specified bilingual terms in translations to ensure accuracy in professional, legal, and medical domains.
• It employs constrained decoding methods like prefix-constrained beam search to guarantee that mandated terms appear correctly in the output.
• The approach integrates synthetic data and backtranslation to improve term fidelity, though strict constraints may sometimes impact fluency.

Terminology-Constrained Machine Translation (TC-MT) refers to a family of methods and modeling strategies for neural or statistical MT that enforce the consistent, explicit use of pre-specified terminology—usually in the form of a bilingual glossary or list of term pairs—within the translation output. This constraint-driven paradigm is critical in professional, legal, medical, and technical domains, where fidelity to authorized term equivalents is as important as overall fluency and adequacy. The challenge is to guarantee that every occurrence of a user-supplied source-side term is rendered by its mandated target-side counterpart, even if that mapping runs counter to corpus- or model-level statistical preferences, and to do so with minimal degradation to grammaticality and fluency.

1. Formal Definition and Constraint Modeling

TC-MT is formally defined in terms of constrained hypothesis spaces in NMT decoding. For a source sentence $x = (x_1, \ldots, x_M)$ and a user-specified set of lexical constraints $C = \{(s_i \rightarrow t_i)\}_{i=1}^K$, where $s_i$ is a source phrase and $t_i$ is the required translation, the constrained hypothesis space is:

$$Y_C(x) = \{ y \in \Sigma^* \;|\; \text{whenever } s_i \text{ occurs in } x, \text{ the string } t_i \text{ occurs as a contiguous subsequence of } y \}$$

The decoding objective under hard constraints is:

$$p(y \mid x, C) \propto \begin{cases} p(y|x) & \text{if } y \in Y_C(x) \ 0 & \text{otherwise} \end{cases}$$

The requirement is that, during search, only hypotheses that meet all terminology constraints are explored, with the strongest variant enforcing verbatim inclusion of all $t_i$ as specified (Burlot, 2019).

2. Constrained Decoding Methodologies

The operationalization of TC-MT is achieved via several constrained decoding algorithms. A central approach is prefix-constrained or grid beam search, as implemented in systems such as Sockeye. At each step, decoding advances a beam of hypotheses, each tagged with a bitmask indicating which constraints have already been fulfilled. Extension of hypotheses involves checking if a next-token candidate extends a pending constraint and updating the bitmask accordingly:

initialize Beam[0][0…2^K−1] = ∅ Beam[0][0] ← { (prefix=[], score=0) } for step = 1 to N_max: for each mask m in 0…2^K−1: for each hyp ∈ Beam[step−1][m]: for each next_token w in top-Vocab(hyp, x): m' = update_mask(m, w, hyp) new_score = hyp.score + log p(w | hyp, x) add (hyp + w, new_score) to Beam[step][m'] prune each Beam[step][*] to beam_size per mask collect completed hyps in Beam[*][(1…1)] and return best

The approach eliminates hypotheses that do not satisfy all constraints by zeroing their probabilities. Recent implementations tag constraint activation per occurrence and integrate named entity recognition to ensure that specific NER types trigger constraints only when matched on both source and target (Burlot, 2019).

3. Synthetic Data, Backtranslation, and Training Procedures

Applying terminology constraints is further enhanced using synthetic parallel data. A typical workflow constructs a large set of monolingual sentences in the target language that are most similar (e.g., via Moore–Lewis score) to the in-domain set, then generates backtranslations under both unconstrained and constrained conditions. In the constrained condition, the backtranslation decoder is forced to insert the required terms at each eligible site, directly creating training triplets that bake in constraint adherence (Burlot, 2019).

For instance, in (Burlot, 2019), 2 million French monolingual sentences were backtranslated into German using (1) standard beam search (unconstrained) and (2) prefix-constrained beam search (constrained), inserting 673,670 constraint tokens. The resulting synthetic pairs are merged with out-of-domain in-domain data (Europarl, News-commentary) to fine-tune the baseline model. This approach is designed to compensate for the lack of in-domain parallel data while inducing constraint-respecting translation behavior in the NMT model.

4. Evaluation Protocols and Empirical Findings

TC-MT system performance is evaluated using both standard and constraint-focused metrics:

• BLEU (SacreBLEU): Measures overall n-gram overlap with references.
• Count of successfully inserted constraints: Tracks the number of required terms actually appearing in output.

Empirical results from (Burlot, 2019) demonstrate:

System	Euelections-dev-2019	Newstest-2013	Newstest-2019
Baseline	25.98	23.48	26.94
+ Constraints	27.87 (+1.89)	23.42 (−0.06)	26.66 (−0.28)

Marginal BLEU improvements are observed in some domains, but gains can be negligible or negative on out-of-domain news sets. However, local term accuracy is improved, especially for named entities and domain-critical expressions ("Alexander Gauland" instead of "Alexandre Gauland"). Qualitative analysis shows improved coverage of entities and political party names, but sometimes at the expense of fluency (Burlot, 2019).

5. Challenges, Limitations, and Future Directions

TC-MT via hard constraints often increases the risk of beam search disruption resulting in ungrammatical or incomplete output. Trade-offs include:

• Hard constraint enforcement can force awkward phrase insertions that break target-language syntax.
• BLEU improvements are not always realized, and sometimes a drop is observed despite perfect constraint satisfaction.
• Constrained backtranslation does not consistently outperform unconstrained backtranslation for BLEU, though it yields quantifiable gains in term-specific accuracy.

The reliability of the terminology lexicon and the ability to extract robust bilingual pairs remain pivotal. Planned improvements include leveraging automated bilingual phrase extraction (phrase-level MUSE, BiLex, unsupervised mining) and the development of soft-constraint approaches that offer more flexibility and less decoder perturbation (Burlot, 2019).

6. Concrete Examples and Illustrative Cases

A representative example from (Burlot, 2019):

Source (French):

Même si les populistes de gauche … comme le montre l’ascension de partis classiques d’opposition tels que Podemos en Espagne et La France Insoumise en France. Constraints: Podemos, France Insoumise Baseline (German):

… klassischen Oppositionsparteien wie Podemos in Spanien und Frankreichs Ununterwürfiges Frankreich … + Constraints:

Podemos in Spanien und France Insoumise in Frankreich … Reference (German):

… klassischen Herausforderer-Parteien wie Podemos … und La France Insoumise …

The constraint-driven system correctly preserves the critical political party names in their original form, matching the gold standard and outcompeting the baseline, which introduces translation artifacts.

---

In summary, terminology-constrained MT methods impose explicit lexical constraints on the output space of NMT decoders in order to ensure term fidelity, especially for named entities and domain-specific expressions. Constrained decoding techniques, especially prefix-constrained beam search, enable rigorous enforcement by operating in the space of constraint-satisfying hypotheses, but evoke nontrivial trade-offs between constraint satisfaction and overall sentence quality. Recent advances combine constraint-aware training or synthetic-data generation with robust inference, but challenges persist in constraint coverage, fluency retention, and ease of lexicon construction. Augmentations such as soft constraints and better bilingual extraction are open areas for further research (Burlot, 2019).