Adaptive-SeqKD for NMT Improvement

• Adaptive-SeqKD is a method that dynamically adjusts teacher and reference contributions during sequence-level knowledge distillation in NMT to mitigate memorization and hallucination.
• It leverages instance-level metrics such as replication rate, extractive memorization, and hallucination signals to set adaptive weights for each training example.
• Experimental results demonstrate a 20–40% reduction in harmful memorization and hallucination patterns while maintaining nearly all translation quality improvements.

Adaptive-SeqKD is a modification of sequence-level knowledge distillation (SeqKD) for neural machine translation (NMT) designed to mitigate the inheritance and amplification of memorization and hallucination behaviors from teacher to student models. Unlike standard SeqKD, which treats all training examples uniformly, Adaptive-SeqKD dynamically adapts the contribution of the teacher and reference targets in the training loss based on instance-level quality or memorization signals. The approach realizes substantial reductions in harmful memorization and hallucination modes while maintaining translation quality (Dankers et al., 3 Feb 2025).

1. Motivation and Background

Sequence-level knowledge distillation (SeqKD) is a standard practice for training compact NMT students by distilling knowledge from a larger teacher model, typically by training the student on the teacher's most likely sequences (beam-search outputs). While this strategy simplifies the distribution the student must model, empirical findings reveal that SeqKD can amplify the teacher's tendencies to memorize atypical or noisy training examples. Specifically, students trained with SeqKD exhibit increased replication rates, higher extractive memorization (ExMem), and elevated frequencies of hallucination—both oscillatory bigram repeats and detached repeats—relative to equally sized baselines trained directly on the WMT human-authored corpus. These findings indicate that SeqKD tightens the teacher-student linkage to an extent that it propagates both beneficial denoising and undesirable memorization/hallucination modes. Adaptive-SeqKD is formulated explicitly to counteract these effects by targeting instance-level adaptation during the distillation process.

2. Adaptive-SeqKD Training Objective

Let $(s_i, t_i) \in \mathcal{D}_C$ denote a parallel corpus instance and $\hat{t}_i = \arg\max_{y} p_{\theta_T}(y \mid s_i)$ the output of the teacher on $s_i$. The vanilla SeqKD loss for the student $\theta_S$ is: $$\mathcal{L}_{\mathrm{SeqKD}} = -\sum_{i \in \mathcal{D}_C} \log p_{\theta_S}(\hat{t}_i \mid s_i)$$ Adaptive-SeqKD introduces a per-example weight $\alpha_i = \alpha(s_i, t_i) \in [0,1]$ that controls the interpolation between the usual SeqKD objective and the cross-entropy loss with respect to the original human target: $$\mathcal{L}_{\mathrm{A\text{-}SeqKD}} = \sum_{i\in\mathcal D_C} \left[\alpha_i\,\mathcal{L}_{\mathrm{KD}}(s_i) + (1-\alpha_i)\,\mathcal{L}_{\mathrm{CE}}(s_i)\right]$$ Here, $$\mathcal{L}_{\mathrm{KD}}(s_i) = -\log p_{\theta_S}(\hat{t}_i \mid s_i)$$ (teacher) and $$\mathcal{L}_{\mathrm{CE}}(s_i) = -\log p_{\theta_S}(t_i \mid s_i)$$ (reference). High $\alpha_i$ values trust the teacher signal; low values override it in favor of the reference, particularly for hallucinated or over-memorized cases.

3. Metrics and Instance Weight Selection

Strategic selection of $\hat{t}_i = \arg\max_{y} p_{\theta_T}(y \mid s_i)$0 relies on multiple instance-level metrics, computed using greedy teacher outputs:

• Replication Rate: Fraction of examples where the model output exactly matches $\hat{t}_i = \arg\max_{y} p_{\theta_T}(y \mid s_i)$1.
• Extractive Memorization (ExMem): Indicates if the model emits $\hat{t}_i = \arg\max_{y} p_{\theta_T}(y \mid s_i)$2 after ingesting only 75% of $\hat{t}_i = \arg\max_{y} p_{\theta_T}(y \mid s_i)$3.
• Natural Hallucination (NatHal): Counts translations that appear at least five times elsewhere among outputs.
• Oscillatory Hallucination (OscHal): Detects bigrams repeated at least 10 times, with high frequency compared to the source.
• Counterfactual Memorization (CM): Influence-based score measuring the model's reliance on $\hat{t}_i = \arg\max_{y} p_{\theta_T}(y \mid s_i)$4 for $\hat{t}_i = \arg\max_{y} p_{\theta_T}(y \mid s_i)$5.

Typically, $\hat{t}_i = \arg\max_{y} p_{\theta_T}(y \mid s_i)$6 is set to $\hat{t}_i = \arg\max_{y} p_{\theta_T}(y \mid s_i)$7 for positive ExMem or detached hallucination signals, $\hat{t}_i = \arg\max_{y} p_{\theta_T}(y \mid s_i)$8 for low CM or high quality (e.g., high COMET-QE), and interpolated otherwise. The paper provides two practical variants: 'random finetuning' (random binary $\hat{t}_i = \arg\max_{y} p_{\theta_T}(y \mid s_i)$9) and 'high-quality finetuning' (binary mask for high-quality subset).

4. Experimental Setup

Experiments use WMT20 parallel corpora for several language pairs (De–En, En–De, Pl–En, En–Pl, Fr–De) and out-of-domain sets (CommonCrawl, Pulpo poetry). Key components:

Model	Architecture	Training Steps
Teacher	Transformer-Large (6x1024, FF=4096)	300k
Student	Transformer-Base (6x512, FF=2048)	100k on SeqKD data
Baseline	Transformer-Base	100k on human data

Baselines include the teacher, vanilla SeqKD student, direct baseline, and Adaptive-SeqKD (random/high-quality finetuning). Evaluation metrics include BLEU, chrF, TER, COMET-20/22 (reference-based), COMET-QE-20/22 (reference-free), and the aforementioned memorization/hallucination rates. The high-quality (HQ) subset comprises ~1M examples with COMET-QE-22 ≥ 0.8.

5. Results and Empirical Analysis

Comparative studies reveal critical distinctions in memorization and hallucination outcomes:

• Replicated Targets: Vanilla SeqKD students show a +3.4% replication rate on WMT20 compared to baselines.
• ExMem Rate: +57% relative ExMem increase under vanilla SeqKD.
• Hallucinations: +31% oscillatory, +14% natural hallucinations versus baseline.
• Adaptive-SeqKD (HQ variant):
  • ExMem reduction: –10 to –30% for 4/5 languages (up to –40% for teacher ExMem).
  • OscHal reduction: –20 to –35% across domains (WMT20, CommonCrawl, Pulpo).
  • NatHal reduction: –8 to –12%.
  • BLEU/COMET shift: ≤1 point, sometimes positive.
• Random finetuning produces only modest ExMem reductions and negligible hallucination mitigation.
• Use of large beam sizes (k=5) for teacher decoding yields additive improvements: OscHal further reduced (–28% on WMT20, –33% on CommonCrawl), NatHal by –10% on WMT20.

Empirical evidence demonstrates that per-instance adaptation—in particular, steering distillation toward high-quality examples—can cut harmful memorization/hallucination modes by approximately 20–40% while preserving nearly all quality improvements achieved by vanilla SeqKD.

6. Practical Variants and Implementation Details

Adaptive-SeqKD can be instantiated using:

• High-quality finetuning: Teacher finetuned on top ~5% COMET-QE-ranked data for 20k–50k steps. Binary $s_i$0: 1 for high-quality, 0 otherwise.
• Random finetuning: Teacher finetuned on a randomly chosen subset; $s_i$1 sampled uniformly from {0,1}.
• Beam size: Increasing teacher decoding beam size ($s_i$2) reduces hallucinations at the expense of a minor BLEU drop (default: $s_i$3, recommended: $s_i$4).
• Temperature: Not used (one-hot sequence-level distillation).
• Student model: Transformer-Base, 100k updates.

Training integrates corpus-based detection of memorization/hallucination signals for $s_i$5 computation and quality monitoring during distillation.

7. Insights, Generalizability, and Recommendations

Recommendations derived from empirical analysis:

• Avoid greedy decoding for knowledge distillation; prefer beam sizes $s_i$6.
• Identify and emphasize a high-quality data slice (top COMET-QE percentile), finetune the teacher on it, and carry the signal through in per-instance adaptive weighting.
• Regularly monitor ExMem, NatHal, OscHal metrics on a held-out set to optimize $s_i$7 thresholds.
• The adaptive per-example weighting framework is applicable to other sequence-generation tasks, provided that cheap, reliable instance-level quality and hallucination metrics are available.

Adaptive-SeqKD operationalizes a dual mitigation strategy: pretraining (teacher finetuning on high-quality data) and in-training (per-instance $s_i$8 interpolation), producing more reliable NMT students without sacrificing standard translation quality (Dankers et al., 3 Feb 2025).