---
title: Language-Aware Token Boosting (LATB)
url: https://www.emergentmind.com/topics/language-aware-token-boosting-latb
type: topic
---

# Language-Aware Token Boosting (LATB)

Language-Aware Token Boosting (LATB) is a decoding-time method for reducing language confusion in large language models without any fine-tuning. It operates by modifying the next-token logits so that tokens associated with a requested target language receive an additive boost, thereby shifting the model’s effective language prior during generation while leaving model parameters frozen. The same work also introduces Adaptive-LATB, which applies the boost only when the model’s own output distribution indicates uncertainty about whether to continue in the target language or in some other language. The method is presented as a tuning-free approach to multilingual alignment, with experiments on multilingual abstractive summarization showing large reductions in language confusion together with preserved or slightly improved ROUGE scores [2606.08994].

## 1. Problem setting and diagnostic criteria

LATB was proposed to address **language confusion**, defined as the failure of an LLM to consistently generate text in the requested language. The reported failure modes are specific: the model may answer in English when instructed to answer in another language, may code-switch within sentences or across lines, or may revert to English for technical terms or entire clauses despite non-English instructions. The paper emphasizes this issue for English-centric models such as Llama3 8B Instruct, where English-dominated pretraining or fine-tuning, shared subword vocabularies, decoding dynamics under sampling, and mixed-language training data can all bias generation toward English [2606.08994].

The work measures language confusion at three granularities. **Token-level language confusion** classifies each output token by language using Unicode ranges and computes the fraction of tokens whose language differs from the target language, averaged over responses. **Line-level language confusion** classifies each line with FastText language identification, computes the proportion of non-target lines for each response, and then averages across responses. **Response-level language confusion** passes the whole response to FastText and computes the fraction of responses whose predicted language does not match the target language. These three metrics are intended to quantify drift at increasingly coarse scales [2606.08994].

The motivation for a decoding-time intervention is practical as well as methodological. The paper positions existing mitigation strategies as predominantly fine-tuning-based: multilingual pretraining, instruction-tuning with language-specific prompts or tags, and RLHF with multilingual preference data. Those approaches can improve language fidelity, but they require access to weights, multilingual data, and nontrivial engineering effort. LATB instead targets **parameter-frozen alignment**, altering only per-step logits during inference [2606.08994].

## 2. Core mechanism of LATB

LATB modifies the model’s logits at each decoding step. If the original model produces logits
$$
\text{logits} \in \mathbb{R}^{|V|},
$$
LATB constructs a perturbation vector $\mathbf{p}$ that boosts tokens associated with the desired language and then replaces the logits with
$$
\text{logits}' = \text{logits} + \mathbf{p}.
$$
Softmax is then applied to obtain the next-token distribution, and decoding proceeds as usual. Because model weights are unchanged, the method is explicitly tuning-free and can be inserted into existing inference stacks [2606.08994].

The vocabulary is partitioned into a desired-language index set
$$
I \subseteq \{1,\dots,|V|\}
$$
and its complement $\bar{I}$. The construction of $I$ is deterministic. For each tokenizer token in string form, the token is considered valid for the target language if all of its characters fall within the Unicode ranges associated with that language, following the Unicode filtering procedure described in the paper. In addition, Arabic numerals and other digits, punctuation and related special characters, and end-of-sentence tokens are included. Formally, if $\mathcal{U}_{\text{lang}}$ denotes the Unicode set for the target language, then a token $t$ is included if
$$
\forall c \in t: c \in \mathcal{U}_{\text{lang}},
$$
or if $t$ is a digit, punctuation, or EOS token. This yields a language-specific token set without using external dictionaries, metadata, or additional models [2606.08994].

Given a boost magnitude $\alpha \ge 0$, the perturbation vector is
$$
\mathbf{p}_i =
\begin{cases}
\alpha & \text{if } i \in I,\\
0 & \text{otherwise.}
\end{cases}
$$
If $\mathbf{z}$ denotes the original logits, LATB computes
$$
\mathbf{z}'=\mathbf{z}+\mathbf{p},
$$
followed by
$$
\mathbf{y}'=\text{Softmax}(\mathbf{z}').
$$
This is the complete “vanilla LATB” procedure: compute logits, add $\mathbf{p}$, apply softmax, and decode from the resulting distribution at every step [2606.08994].

A central theoretical property is that additive logit perturbation preserves relative probabilities *within* the boosted subset. If $m,n \in I$, then
$$
\frac{y'_m}{y'_n}=\frac{y_m}{y_n}.
$$
The perturbation therefore does not alter the model’s ranking among target-language tokens; it alters only the relative probability mass assigned to the boosted set versus the non-boosted set. The paper uses this to argue that conditional language modeling quality within the target language is preserved while cross-language competition is shifted toward the requested language [2606.08994].

LATB is also decoding-strategy agnostic. The modified logits can be used with greedy decoding, temperature and top-$p$ sampling, or beam search. In practical implementations such as Hugging Face, the method is expressed as a `LogitsProcessor` that receives `logits: Tensor[batch, vocab_size]`, adds $\alpha$ to the positions in $I$, and returns the modified tensor. The target language is supplied explicitly by the user or application rather than inferred automatically [2606.08994].

## 3. Adaptive-LATB and confidence-gated perturbation

Adaptive-LATB was introduced because always boosting the target-language token set can be too rigid. The motivating case is that a response in a non-English language may still require English strings such as product names, acronyms, or otherwise untranslatable expressions. A large constant boost can then over-constrain generation and yield unnatural output, including transliteration or forced rendering of expressions that should remain in English [2606.08994].

To address this, Adaptive-LATB computes a confidence signal directly from the base model’s output distribution. Let
$$
\mathbf{y}=\text{Softmax}(\mathbf{z})
$$
for original logits $\mathbf{z}$. Define
$$
a=\max\{y_i \mid i \in I\},
\qquad
b=\max\{y_i \mid i \notin I\}.
$$
Here, $a$ is the highest probability among target-language tokens and $b$ is the highest probability among non-target-language tokens. A confidence-difference threshold $\beta \in [0,1]$ is then introduced. If
$$
|a-b|<\beta,
$$
the model is treated as not being strongly committed to either side, and the perturbation is applied. Otherwise, no perturbation is applied and the original distribution is used [2606.08994].

The adaptive procedure is therefore:
1. compute logits,
2. compute softmax probabilities,
3. compute $a$ and $b$ from the target-language and non-target-language subsets,
4. apply $\mathbf{p}$ only if $|a-b|<\beta$,
5. decode from the resulting distribution.

The paper stresses that the confidence signal is entirely internal to the LLM’s own token distribution; no external classifier is introduced. Because the decision is made at every generation step, the method can leave non-target tokens untouched when the model is highly confident they are appropriate, while still intervening when the model is effectively “on the fence” between languages [2606.08994].

The distinction between LATB and Adaptive-LATB is therefore not architectural but operational. Vanilla LATB is a fixed logit shift. Adaptive-LATB is a conditional logit shift driven by the instantaneous separation between the most probable target-language token and the most probable non-target token. This suggests that Adaptive-LATB is a more selective control mechanism over the same language-specific token set.

## 4. Empirical evaluation and observed behavior

The evaluation centers on **multilingual abstractive summarization** on XL-Sum across eight languages: Russian, Simplified Chinese, Japanese, French, Korean, Thai, Hindi, and Arabic. Up to 1,000 test instances were used per language. The primary base model was **Llama3 8B Instruct**, described in the paper as English-centric; the fine-tuned multilingual baseline was **Suzume 8B Multilingual**; and an appendix also evaluates **Qwen3 4B-Instruct**, which is described as having strong multilingual alignment out of the box. All methods used temperature $=1.0$ and top-$p=1.0$. The main hyperparameters were $\alpha=5$ for LATB and $\alpha=1000$, $\beta=0.8$ for Adaptive-LATB [2606.08994].

The reported reductions in language confusion are large. For Llama3 8B Instruct with standard prompts, non-Latin scripts showed very high confusion rates, including Russian at **80.66 / 93.83 / 92.50**, Chinese at **85.92 / 98.90 / 98.90**, Korean at **85.46 / 99.60 / 99.63**, Thai at **86.03 / 99.80 / 99.39**, Hindi at **86.18 / 99.05 / 98.50**, and Arabic at **86.21 / 99.27 / 98.30** for token-, line-, and response-level confusion respectively. With LATB, these figures fell sharply: Russian became **0.28 / 0.44 / 0.10**, Thai **0.43 / 0.18 / 0.00**, Hindi **0.23 / 0.74 / 0.10**, and Arabic **0.37 / 0.28 / 0.00**. Adaptive-LATB yielded similarly low confusion, with slightly higher or similar token-level values and extremely low line- and response-level confusion. Strict prompts improved over standard prompting, but remained markedly worse than LATB; for Russian, strict prompting yielded **5.02 / 4.10 / 2.90** [2606.08994].

The same experiments report preserved or slightly improved summarization quality under ROUGE. For Chinese, the highly confused base setting scored **0.80 / 0.32 / 0.69** on ROUGE-1 / ROUGE-2 / ROUGE-L. Strict prompting improved this to **19.41 / 8.99 / 13.73**, Suzume 8B achieved **19.31 / 8.59 / 13.38**, LATB reached **20.70 / 9.44 / 14.64**, and Adaptive-LATB **20.55 / 9.28 / 14.52**. For Russian, strict prompting scored **20.44 / 9.26 / 13.41**, Suzume **19.35 / 8.32 / 12.42**, LATB **20.83 / 9.46 / 13.60**, and Adaptive-LATB **21.00 / 9.42 / 13.58**. Similar patterns are reported for Korean, Thai, Hindi, and Arabic [2606.08994].

The appendix reports two further results that shape interpretation. First, performance improvement correlates strongly with initial language confusion: tasks where the base model is more confused benefit more from LATB. Second, on a strongly multilingual model such as Qwen3 4B-Instruct, base confusion was already reported as **0%**; LATB and Adaptive-LATB maintained **0%** confusion and did not degrade ROUGE-L. The paper presents this as consistent with the theoretical claim that within-language token rankings are preserved [2606.08994].

Throughput measurements on vLLM with an A100 indicate negligible overhead for vanilla LATB and moderate overhead for the adaptive variant. Reported speeds were **1145.8 tokens/s** for base Llama3 8B Instruct, **1189.5 tokens/s** for Llama3 with LATB, and **838 tokens/s** for Llama3 with Adaptive-LATB. The reported explanation is that Adaptive-LATB requires an additional softmax and max operations at every step [2606.08994].

## 5. Implementation profile, hyperparameters, and usage conditions

LATB is applied directly after the model’s final linear projection and before softmax. It is invoked at every decoding step and for every partial sequence, including each beam in beam search. In Hugging Face-style implementations, the method is described as a `LogitsProcessor` or `LogitsWarper`; the essential operation is adding $\alpha$ to the positions corresponding to the language-specific token set $I$ [2606.08994].

The paper reports a clear hyperparameter trade-off. For LATB, experiments varied $\alpha$ from **0 to 50**. As $\alpha$ increases, confusion decreases monotonically, but ROUGE first improves and then degrades once non-target tokens are overly suppressed. For Adaptive-LATB, the threshold $\beta$ was tested from **0 to 0.9**; increasing $\beta$ increases the frequency of boosting and lowers confusion up to a point, but values that are too high can restrict necessary non-target tokens and cause a mild ROUGE drop. The main experiments selected $\alpha=5$ for LATB and $\alpha=1000$, $\beta=0.8$ for Adaptive-LATB [2606.08994].

The paper’s practical guidance is correspondingly differentiated. LATB is recommended when the target language is script-distinct, when the simplest and fastest method is preferred, and when a small grid search over $\alpha$ is acceptable. Adaptive-LATB is recommended when the model may need to retain foreign-language expressions, when robustness to hyperparameters is desirable, and when a modest throughput penalty is acceptable. For new settings, the suggested workflow is: identify the target language and its Unicode ranges, build $I$ by scanning the tokenizer vocabulary, start LATB around $\alpha \in [2,10]$, and switch to Adaptive-LATB with large $\alpha$ and $\beta \in [0.6,0.9]$ if needed non-target words are being unnaturally suppressed [2606.08994].

A recurrent misconception is that LATB is equivalent to hard constrained decoding or vocabulary filtering. The paper explicitly distinguishes it from those methods. LATB is a **soft** intervention: it biases the target-language token set without forbidding other tokens. It is also more structured than generic logit biasing because the boosted set is derived automatically from Unicode-based token classification and, in the adaptive variant, the intervention is gated by the model’s own confidence state [2606.08994].

## 6. Scope, limitations, and relation to adjacent research

The paper is explicit that LATB does not create missing linguistic competence. If the base model has not learned a language, boosting tokens for that script cannot “conjure abilities” the model does not possess. The Unicode-based token association works best for script-distinct languages such as Thai, Arabic, Hindi, Russian, Japanese, and Chinese, but is less effective for languages sharing the Latin alphabet, because the token set overlaps heavily and language control becomes weaker. Mixed tokens containing both Latin and non-Latin characters can also be misclassified. Vanilla LATB is especially sensitive to overly large $\alpha$, which can lead to unnatural output and forced transliteration. Adaptive-LATB reduces but does not eliminate these trade-offs, and it incurs additional per-step computation [2606.08994].

These limitations place LATB within a broader family of token-aware and language-aware interventions. Related work has explored language-specific output heads and vocabularies for excessively tokenized languages, keeping the transformer backbone frozen while adapting the head for faster generation [2401.10660]. Other work has proposed language-adaptive tokenization from a single fixed shared vocabulary by learning language-specific unigram distributions and selecting the highest-likelihood segmentation at inference time, again treating token probabilities in a language-conditioned way [2606.23566]. The same general design space includes LM-aware speech tokenization, where a frozen text LM’s next-token objective shapes discrete speech units [2409.03701], and token-aware, layer-localized contrastive decoding, where token categories are reweighted through internal perturbations at specific transformer depths [2507.04404]. This suggests a broader interpretation of LATB as one member of a larger class of token-distribution interventions that alter generation behavior without full model retraining.

Within that broader landscape, the distinctive contribution of LATB is its specific combination of properties: it is decoding-time rather than training-time, parameter-frozen rather than fine-tuned, language-targeted rather than task-generic, and implemented through direct logit perturbation over a Unicode-derived token set. Its empirical results indicate that, for multilingual generation with script-distinct target languages, a simple additive perturbation can reduce language confusion from very high baseline levels to near zero response-level confusion while preserving summarization quality [2606.08994].

Source: https://www.emergentmind.com/topics/language-aware-token-boosting-latb