SafeMoE: Preserving Safety in MoE LLM Fine-Tuning

Updated 12 July 2026

SafeMoE is a safety fine-tuning approach for Mixture-of-Experts LLMs that preserves safety-critical routing patterns using KL divergence regularization.
It mitigates harmful fine-tuning by restraining routing drift, achieving a marked reduction in harmfulness scores with minimal utility loss.
SafeMoE employs a Greedy Bi-Level optimization that alternates between standard fine-tuning and periodic safety regularization on curated harmful instructions.

SafeMoE is a safe fine-tuning method for Mixture-of-Experts (MoE) LLMs that targets a specific architectural failure mode: the drift of harmful-input routing away from the safety-aligned routing pattern established in the original model. In the formulation introduced in "Defending MoE LLMs against Harmful Fine-Tuning via Safety Routing Alignment" (Kim et al., 26 Sep 2025), safety in MoE LLMs depends not only on parameter values but also on whether harmful inputs continue to activate safety-critical experts after downstream fine-tuning. SafeMoE therefore regularizes fine-tuning so that the routing weights for harmful instructions remain close to those of the initial safety-aligned model, with the stated aim of preserving refusal behavior and other safety-aligned responses under both benign fine-tuning and harmful fine-tuning attacks.

1. Architectural premise: safety as a routing property

In an MoE layer, each token is processed by a gating network that produces a weight vector over experts, selects the top- $k$ experts, and aggregates their outputs. Within the SafeMoE account, this routing mechanism is not merely a computational optimization; it is also part of the model’s safety apparatus. Harmful inputs in the safety-aligned model are routed to specific safety-critical experts, and those experts are described as essential for filtering or refusing unsafe requests (Kim et al., 26 Sep 2025).

This framing departs from the usual assumption that alignment is encoded only in the global parameter state of a Transformer. For MoE LLMs, the paper argues that safety relies on a "superficial safety mechanism" in which harmful inputs are routed to the experts responsible for safe handling. The paper further states that masking or disabling these safety experts, even without changing other weights, can significantly degrade model safety, implying that correct input-to-expert routing is a first-order safety requirement rather than an incidental by-product of training (Kim et al., 26 Sep 2025).

A central implication is that MoE safety is structurally conditional. The same base model can preserve or lose safe behavior depending on whether harmful queries continue to traverse the expert pathways that implement refusal or safe redirection. This suggests that, for MoE systems, alignment analysis must include router behavior and expert activation patterns, not only output-level behavior or parameter deltas.

2. Safety routing drift and harmful fine-tuning

SafeMoE is motivated by the claim that fine-tuning substantially and consistently alters the routing of harmful inputs. The paper terms this phenomenon safety routing drift and defines it as the KL-divergence between the routing weights of the original safety-aligned model and those of the fine-tuned model on a harmful input $x$ : $d(M_{ft}, x) = D_{KL}\big( \sigma(g(x|_{align})) \Vert \sigma(g(x|_{ft})) \big).$ Here, $M_{align}$ denotes the safety-aligned model and $M_{ft}$ the fine-tuned model (Kim et al., 26 Sep 2025).

The reported empirical finding is that routing drift and harmfulness are tightly coupled. In the paper’s Figure 1, increased routing drift is said to correlate strongly with increased harmfulness scores, with $r > 0.9$ , and this occurs even after fine-tuning on benign data alone. The same summary states that this coupling means routing drift is both necessary and sufficient for the breakdown of MoE model safety (Kim et al., 26 Sep 2025).

This diagnosis is used to explain the vulnerability to harmful fine-tuning (HFT) attacks. According to the paper, injecting a small proportion of harmful samples during downstream fine-tuning can produce a high rate of unsafe outputs because the harmful-input routes drift away from the original safety-critical experts. The problem is presented as MoE-specific: existing defenses such as SafeInstr, SaLoRA, Antidote, and SafeDelta were designed for monolithic Transformer LLMs and do not directly prevent harmful-input routing drift in sparse expert architectures (Kim et al., 26 Sep 2025).

A common misconception is that safety loss after fine-tuning is primarily a matter of catastrophic forgetting in the dense-model sense. SafeMoE instead centers routing instability as the proximate mechanism of safety failure in MoE LLMs. On this account, restoring or preserving safe outputs requires preserving the expert-selection pattern for harmful inputs, not only constraining the global weight update.

3. SafeMoE objective and optimization procedure

SafeMoE introduces a fine-tuning-time regularizer applied to a preselected harmful-instruction set $\mathcal{D}_h$ . For each harmful input and each selected transformer layer, the method penalizes divergence between the routing distribution of the current model and that of the initial safety-aligned model: $\mathcal{L}_{reg}(M) = \mathbb{E}_{x \in \mathcal{D}_h} \mathbb{E}_{l \in L} D_{KL}\left(\sigma(g^{(l)}(x|_{align})/\tau) \Vert \sigma(g^{(l)}(x|_M)/\tau)\right).$ The temperature $\tau$ controls how sharply the regularizer focuses on the most important experts; smaller values place greater emphasis on top experts (Kim et al., 26 Sep 2025).

The full fine-tuning objective is

$\min_M \left[ \mathcal{L}_{sft}(M) + \mathcal{L}_{reg}(M) \right].$

In the paper’s interpretation, this directly penalizes routing divergence on harmful inputs and thereby preserves activation of safety-critical experts during downstream adaptation (Kim et al., 26 Sep 2025).

For efficiency, SafeMoE uses what the paper calls “Greedy Bi-Level” optimization. Rather than jointly optimizing task loss and routing regularization at every step, it alternates between standard fine-tuning on task data and periodic application of the routing loss on harmful samples every $x$ 0 steps. The routing weights of the initial model on the harmful prompts are precomputed. This is intended to preserve the effect of the regularizer while avoiding a large computational penalty (Kim et al., 26 Sep 2025).

Several implementation features are emphasized. The reported default uses 100 harmful instructions for regularization. Regularization may be applied to all transformer layers or only a selected subset, especially the upper layers, where routing drift is reported to be most severe. The method is stated to work for both LoRA and full fine-tuning (Kim et al., 26 Sep 2025).

4. Experimental profile and reported results

The experimental study covers 8 open-source safety-aligned MoE LLMs ranging from OLMoE-7B to Mixtral-8x22B/141B, including GPT-OSS, Llama 4, and Qwen3. The downstream tasks are dialog summarization (SAMSum) and SQL query generation, with HFT attacks constructed from 500 harmful + 5k task-specific samples. Utility is measured with Rouge/exact match and MMLU-Redux for reasoning, while safety is measured by a harmfulness score defined as the proportion of unsafe outputs on JailbreakBench, classified by Llama Guard 4-12B (Kim et al., 26 Sep 2025).

The headline result in the abstract is that SafeMoE mitigates HFT attacks while maintaining task utility within 1% degradation and incurring only 2% overhead. The detailed summary reports 2.13% extra training time relative to vanilla fine-tuning (Kim et al., 26 Sep 2025).

A representative subset of the reported numbers is shown below.

Setting	Vanilla fine-tuning	SafeMoE
OLMoE-7B on SAMSum, harmfulness	62.0	5.0
OLMoE-7B on SAMSum, utility	49.3	48.9
GPT-OSS, harmfulness	84.0	7.0

For OLMoE-7B on SAMSum, the initial aligned model is reported to have harmfulness score 0, vanilla fine-tuning raises this to 62.0, and SafeMoE reduces it to 5.0; the best baseline is reported at 13.0. On the same example, utility changes from 49.3 under vanilla fine-tuning to 48.9 under SafeMoE. For GPT-OSS, harmfulness is reported to decrease from 84.0 under fine-tuning to 7.0 under SafeMoE, with only minor impact on MMLU reasoning (Kim et al., 26 Sep 2025).

The paper’s Figure 2 is described as showing that SafeMoE quickly reduces routing drift after the first regularization checkpoint, accompanied by a sharp drop in harmfulness and negligible impact on fine-tuning loss. Figure 3 reportedly shows that applying regularization to only the upper half of transformer layers achieves nearly full mitigation, enabling further speed-up. Figure 4 reports that lower $x$ 1 improves safety with negligible accuracy loss, that using more harmful instructions for regularization improves safety, and that SafeMoE remains robust as the ratio of harmful samples in the fine-tuning data increases (Kim et al., 26 Sep 2025).

5. Position within MoE safety research

SafeMoE belongs to a broader shift toward architecture-aware safety methods for sparse models. The immediate contrast drawn in the paper is with SafeInstr, SaLoRA, Antidote, and SafeDelta, which are presented as monolithic-model defenses that do not preserve harmful-input routing in MoE systems (Kim et al., 26 Sep 2025). Related work reinforces the claim that MoE safety is inseparable from routing structure, expert specialization, and sparse architectural failure modes.

MoE reliability benchmarking had already indicated that safety and robustness in sparse models are sensitive to MoE-specific design choices such as router training, expert dropout, and balancing losses, and that appropriate training and inference protocols can make SMoEs as reliable as, or more reliable than, dense models (Chen et al., 2024). Subsequent MoE-specific analyses made the dependence on narrow safety pathways more explicit. SAFEx formalizes positional vulnerability, identifies safety-critical experts with a Stability-based Expert Selection (SES) algorithm, and reports that in Qwen3-30B-A3B, disabling 12 identified safety-control experts reduces refusal rate from 93.6% to 71.6%, a drop of 22.0% (Lai et al., 20 Jun 2025). "Sparse Models, Sparse Safety" extends the argument to routers, introducing RoSais and F-SOUR to expose unsafe routes; it reports that masking 5 routers in DeepSeek-V2-Lite raises attack success rate on JailbreakBench to 0.79, and that F-SOUR reaches average ASR 0.90 on JailbreakBench and 0.98 on AdvBench across four MoE families (Jiang et al., 9 Feb 2026). RASA, in turn, argues that naïve full-parameter alignment can reduce attack success rates through routing or expert dominance rather than by repairing unsafe experts; its remedy is to identify Safety-Critical Experts, fine-tune only those experts under fixed routing, and then enforce router consistency with safety-aligned contexts (Liang et al., 4 Feb 2026). Outside the MoE-specific line, SOMF addresses post-fine-tuning safety degradation through subspace-oriented model fusion and learnable safety subspace masking, but it is formulated for safety realignment and fusion in general LLM settings rather than as a routing-preservation method for MoE architectures (Yi et al., 2024).

These strands can be summarized as follows.

Method or line	Primary safety locus	Characteristic intervention
SOMF	Parameter subspace	Safety subspace masking and model fusion
SafeMoE	Harmful-input routing	KL regularization to preserve safety-aligned routes
SAFEx	Safety-critical experts	Stable expert identification and masking analysis
RASA	Safety-critical experts and router	Selective expert repair under fixed routing, then router consistency
Unsafe Routes	Safety-critical routers	Route discovery and router manipulation analysis

Taken together, the literature positions SafeMoE as one member of a larger class of MoE-native safety techniques. Its distinctive claim is that preserving the original routing distribution for harmful inputs is sufficient to recover most of the lost safety under fine-tuning, whereas adjacent work emphasizes expert auditing, router auditing, or selective expert repair.

6. Limitations, scope, and implications

The SafeMoE paper notes several limitations and implementation constraints. The method requires a small harmful-instruction set $x$ 2 for regularization, with a default of 100 instructions from SafeInstr or similar. It also requires modest tuning of the temperature hyperparameter $x$ 3 to balance safety and utility: lower $x$ 4 strengthens safety, but if set too low it may reduce utility. For efficiency, routing weights of the initial model on all harmful prompts are precomputed. The method is reported as currently tested with standard fine-tuning and LoRA (Kim et al., 26 Sep 2025).

A more substantive limitation is coverage. The paper states that SafeMoE is only as robust as the harmful-instruction set used for regularization, and that if future harmful instructions differ substantially, safety may partially degrade; the summary characterizes this as a general limitation for blacklist-based methods (Kim et al., 26 Sep 2025). This suggests that SafeMoE is best understood as a targeted fine-tuning defense against routing drift on a curated harmful set, rather than as a complete guarantee against arbitrary future attack distributions.

The broader MoE safety literature points to complementary directions. SAFEx recommends position-aware safety alignment, functional redundancy, robust routing, and architectural auditing to reduce reliance on a small set of experts (Lai et al., 20 Jun 2025). RASA argues for targeted expert repair rather than global parameter updates, explicitly to avoid routing-based shortcut solutions (Liang et al., 4 Feb 2026). In that context, SafeMoE can be read as a routing-preservation approach that addresses one central failure mode of MoE fine-tuning: safety degradation caused by harmful-input route drift.

Its significance lies in turning routing from an unobserved internal variable into an explicit alignment target. For dense-model safety methods, regularization typically constrains outputs, parameters, or preference losses. SafeMoE instead operationalizes the claim that in sparse expert models, the route taken by harmful inputs is itself part of the safety contract (Kim et al., 26 Sep 2025).