Smoothie-Qwen: Post-hoc Token Smoothing

• Smoothie-Qwen is a method that post-processes token probabilities to reduce language bias in multilingual Qwen models.
• It identifies risky Chinese tokens via deterministic and heuristic risk scoring to selectively adjust the lm_head weights.
• Empirical results show over 95% suppression of unintended Chinese output while maintaining task performance on Korean benchmarks.

Smoothie-Qwen is a post-hoc method for reducing language confusion in multilingual LLMs, demonstrated on Qwen, by selectively adjusting token-level output probabilities in the final output layer rather than retraining the model. In the supplied literature, the name also appears in a broader, more informal sense as a label for Qwen-based system designs, but its most specific and technically defined meaning is the language-bias mitigation method introduced for Qwen2.5-Coder-14B-Instruct, where token-specific smoothing is applied to suppress unintended Chinese generation while preserving downstream task performance (Ji et al., 8 Jul 2025).

1. Naming, scope, and distinction from other “Smoothie” methods

Within the provided corpus, “Smoothie-Qwen” has a precise sense and a broader systems-design sense. The precise sense is the post-hoc smoothing method for multilingual language control in Qwen (Ji et al., 8 Jul 2025). Separately, several Qwen-oriented design notes use “Smoothie-Qwen” as a generic system label for hypothetical or prospective Qwen-based architectures, especially in multimodal retrieval, creative writing, and reasoning settings (Li et al., 8 Jan 2026).

This dual usage matters because the root term “Smoothie” is also used by unrelated methods. “Smoothie: Label Free LLM Routing” defines an unsupervised routing method based on a latent-variable graphical model over output embeddings (Guha et al., 2024). “Smoothie: Smoothing Diffusion on Token Embeddings for Text Generation” defines a diffusion framework over token-embedding-induced distance profiles for text generation (Shabalin et al., 24 May 2025). These methods are conceptually separate from Smoothie-Qwen’s output-layer debiasing.

Name in the literature	Primary object	Core mechanism
Smoothie-Qwen	Multilingual Qwen output control	Post-hoc lm_head smoothing
Smoothie	LLM routing	Label-free routing from output embeddings
Smoothie	Text diffusion	Diffusion on token-embedding-induced similarity profiles

A common misconception is that Smoothie-Qwen is a retrained multilingual model or a runtime language-routing system. It is neither. The defining method is a one-time modification of the model’s final projection weights, applied after training, with no architectural change and no gradient-based retraining (Ji et al., 8 Jul 2025).

2. Problem setting: language confusion in multilingual Qwen

The motivating problem is language confusion: a multilingual LLM tends to answer in a dominant training language even when the prompt is in another language. In the Smoothie-Qwen study, the dominant undesired language is Chinese, and the model under study is Qwen2.5-Coder-14B-Instruct (Ji et al., 8 Jul 2025).

The underlying explanation given in the paper is distributional rather than architectural. Training data are imbalanced across languages, and next-token training reinforces dominant-language continuations. As a result, Chinese tokens can receive high probability even in Korean-language interaction settings. The operational consequence is that a Korean prompt may elicit Chinese output, or mixed-language output, despite no explicit user request for Chinese (Ji et al., 8 Jul 2025).

Smoothie-Qwen addresses this as a controllability problem at the token-distribution level. Rather than introducing prompt-only constraints or retraining with new multilingual supervision, it edits the final linear projection layer, lm_head, so that tokens associated with the undesired language become less likely. This is framed as a lightweight and practical intervention for already released Qwen checkpoints (Ji et al., 8 Jul 2025).

The method is designed to suppress unintended Chinese output while retaining task competence. It is not designed as a general language-selection controller. This suggests a narrower operational goal: reducing a specific form of multilingual bias, not solving multilingual generation policy in full generality.

3. Methodology: risk scoring and post-hoc smoothing of lm_head

The method has three stages: identifying risky tokens, assigning token-level risk scores, and scaling lm_head weight vectors according to those scores (Ji et al., 8 Jul 2025).

The vocabulary is partitioned into three categories. First, target tokens are tokens directly belonging to the undesired language. For Chinese, these are detected using Unicode ranges such as U+4E00 to U+9FFF. Second, broken tokens are token fragments that do not themselves look like Chinese characters but can combine into them under the tokenizer. Third, all remaining tokens are treated as safe. On Qwen2.5-Coder-14B-Instruct, the paper reports a vocabulary of 151,643 tokens, with 26,153 target Chinese tokens (17.25%) and 1,457 broken tokens (0.96%) (Ji et al., 8 Jul 2025).

Risk assignment is deterministic for direct target tokens and heuristic for broken tokens. If a token belongs to the target set, its risk score is 1.0. If it belongs to the broken-token set, the risk score is estimated by N-gram sampling: random token combinations involving that fragment are decoded, and the fraction that yield valid Chinese characters becomes the token’s estimated risk. Tokens outside both sets receive risk score 0.0 (Ji et al., 8 Jul 2025).

The smoothing step maps risk score $r \in [0,1]$ to a scale factor $S \in (0,1]$ using the paper’s non-linear function:

$$S = 1 - (1 - \text{min\_scale}) \times \frac{\log\left(1 + (\text{smoothness} - 1) \times \text{risk\_score} \right)}{\log(\text{smoothness})}$$

This mapping has two boundary properties emphasized in the paper. If $r=0$, then $S=1$, so safe tokens are unchanged. If $r=1$, then $S=\text{min\_scale}$, so maximally risky tokens are down-weighted to the chosen floor (Ji et al., 8 Jul 2025).

The actual parameter edit is simple:

$w'_t = S \cdot w_t$

where $w_t$ is the token-specific weight vector in lm_head. This reduces the corresponding token’s logit contribution in typical settings. The paper explicitly notes a caveat: if a token’s logit is negative, scaling the weight could move the logit closer to zero and thereby increase its relative probability. The authors argue empirically that high-risk Chinese tokens typically have positive logits in the contexts where they are considered, so the practical side effect is negligible (Ji et al., 8 Jul 2025).

Two hyperparameters govern the intervention. min_scale sets the minimum retained weight magnitude for maximally risky tokens, and smoothness controls how aggressively medium-risk tokens are suppressed. Because $S$ never reaches zero, Smoothie-Qwen is a soft suppression method rather than a hard ban. This distinction is important for interpreting later examples, where Chinese characters may disappear but pinyin or Unicode descriptions remain (Ji et al., 8 Jul 2025).

4. Empirical evaluation and observed behavior

The evaluation combines a custom Chinese-elicitation dataset and Korean-language knowledge benchmarks. The custom dataset contains 1,000 prompt-response pairs, with Korean prompts that explicitly ask for Chinese translations across 26 categories such as numbers, food, sports, and family. The second evaluation source is KMMLU, specifically the Computer Science and Industrial Engineering domains, used in both generative and multiple-choice settings (Ji et al., 8 Jul 2025).

The principal suppression metric on the elicitation set is chin_prom, while chin_cs and chin_ie measure unintended Chinese generation in KMMLU’s generative setting, and acc_cs and acc_ie measure multiple-choice accuracy. Without smoothing, the baseline model has chin_prom = 0.190, chin_cs = 0.995, chin_ie = 0.990, acc_cs = 0.715, and acc_ie = 0.385. With smoothness = 10.0, decreasing min_scale steadily raises suppression on the elicitation benchmark: at min_scale = 0.5, chin_prom = 0.950, while acc_cs = 0.710 and acc_ie = 0.395 remain essentially unchanged (Ji et al., 8 Jul 2025).

A second ablation fixes min_scale = 0.5 and varies smoothness. The paper reports chin_prom = 0.920 at smoothness = 1.1, 0.950 at 10.0, 0.990 at 100.0, and 1.000 at 1000.0, again with acc_cs = 0.710 and acc_ie = 0.395 throughout. The paper therefore recommends min_scale = 0.5 and smoothness = 10.0 as a practical operating point, yielding over 95% suppression of unintended Chinese output with negligible change on Korean KMMLU accuracy (Ji et al., 8 Jul 2025).

The qualitative examples clarify what “soft suppression” looks like. When asked for the numbers 1 to 3 in Chinese, the unsmoothed model outputs Chinese characters such as “一, 二, 三”, while Smoothie-Qwen outputs romanized forms such as “yī, èr, sām” without the Chinese characters. In code-oriented prompts, the smoothed model often replaces literal Chinese characters with Unicode numeric ranges like 0x4E00 to 0x9FFF. When explicitly asked to add comments in Chinese, the smoothed model tends to avoid full Chinese comments and may instead produce English or mixed alternatives, indicating that the smoothing acts globally rather than conditionally (Ji et al., 8 Jul 2025).

These results support two technical interpretations. First, the method is highly effective at changing token preference profiles in a targeted way. Second, because accuracy on the reported KMMLU tasks remains flat, the intervention appears to perturb language-specific output behavior more than task-solving competence. A plausible implication is that much of the desired task knowledge resides upstream of the final token choice, while the smoothing primarily reshapes the surface realization layer.

5. Position within the broader Qwen research landscape

In the broader literature provided, “Smoothie-Qwen” also functions as an informal design label for prospective Qwen-based systems. One write-up uses the name for a Qwen3-VL-based multimodal retrieval-and-reranking stack built from Qwen3-VL-Embedding and Qwen3-VL-Reranker, emphasizing unified multimodal embeddings, Matryoshka Representation Learning, quantization-aware training, and cross-encoder reranking (Li et al., 8 Jan 2026). Another body of work discusses Qwen-centered evaluation and optimization themes that are directly relevant to such systems, even when the name is only illustrative.

This broader usage intersects with several adjacent Qwen research directions. In Chinese film script continuation, a Qwen-based creative-writing system would face deficits in character consistency, tone-style matching, structural preservation, and generation stability relative to GPT-5.2, despite slightly higher ROUGE-L on paired samples (Cao et al., 21 Jan 2026). In vision-language reasoning, Qwen-LookAgain addresses visual token dilution during long reasoning by adding reflection segments with visual re-attention via token copy or routing (Chu et al., 29 May 2025). In tokenizer-free language adaptation, KazByte proposes a byte-level adapter around frozen Qwen2.5-7B to bypass tokenizer tax for Kazakh and then fine-tune only attention layers (Akylzhanov, 29 Mar 2026).

Other adjacent directions include preference-aligned multimodal safety classification and distilled reasoning systems. QwenSafe adapts Qwen3-VL-8B to identify Apple content rating descriptors from metadata and screenshots via supervised fine-tuning followed by DPO, improving positive-class recall over multiple baselines (Denipitiyage et al., 20 May 2026). “Thinking with DistilQwen” extends DistilQwen into slow-thinking, adaptive-thinking, and distilled reward-model series for industrial reasoning workloads (Cai et al., 3 Nov 2025). Read together, these works suggest that “Smoothie-Qwen” in the broader ecosystem denotes not a single architecture, but a family of Qwen-centered interventions for controllability, retrieval, reasoning, adaptation, and policy alignment.

That broader ecosystem also explains why confusion with other “Smoothie” methods is common. Label-free routing via Smoothie (Guha et al., 2024) and diffusion-based text generation via Smoothie (Shabalin et al., 24 May 2025) are orthogonal techniques that could, in principle, be combined with Qwen models, but they are not the same method as Smoothie-Qwen’s lm_head smoothing.

6. Limitations, misconceptions, and future directions

The principal limitation of Smoothie-Qwen is that the smoothing is static. Once applied, it is always active, and the model cannot distinguish unintended language drift from legitimate requests that genuinely require Chinese. The paper explicitly frames this as a context-insensitivity problem: a request to translate into Chinese and a request to answer in Korean are both processed by the same altered output layer (Ji et al., 8 Jul 2025).

A second limitation is the heuristic nature of risk estimation for broken tokens. Direct Unicode-matched Chinese tokens receive exact risk 1.0, but broken-token risk is estimated by N-gram sampling rather than full combinatorial analysis or corpus statistics. This can induce both false negatives and false positives in token risk assignment (Ji et al., 8 Jul 2025).

A third limitation is hyperparameter dependence. The trade-off between suppression strength and retained flexibility is governed by min_scale and smoothness, and the paper states that optimal values are model- and language-specific. This means that transfer to other base models, other undesired languages, or other deployment settings is not calibration-free (Ji et al., 8 Jul 2025).

Several misconceptions follow naturally and are directly contradicted by the method. Smoothie-Qwen does not retrain Qwen; it edits lm_head post hoc. It does not hard-ban Chinese; it down-weights risky token vectors and keeps $S \in (0,1]$0. It is not a context-aware multilingual policy engine; it is a static token-preference transformation. These distinctions matter when comparing it with more elaborate Qwen-based systems such as QwenSafe’s descriptor-aware multimodal alignment (Denipitiyage et al., 20 May 2026) or DistilQwen’s curriculum, DPO, and reward-model pipelines for reasoning (Cai et al., 3 Nov 2025).

The future directions proposed in the paper are correspondingly targeted. They include context-aware dynamic smoothing, improved risk scoring using corpus-based statistics, automated hyperparameter search such as Bayesian optimization, and extensions from language bias to other token-level suppression targets such as profanity, slurs, and stylistic controls (Ji et al., 8 Jul 2025). This suggests that Smoothie-Qwen is best understood as a minimal but effective instance of a larger class of post-hoc output-layer control methods for multilingual Qwen models.