Routing-Drift in MoE and Continual Learning

Updated 16 July 2026

Routing-Drift is an umbrella term for failures in mapping inputs to intended experts, arising from sequential updates, distribution shifts, and hardware degradation.
It manifests as phenomena like semantic drift, misaligned co-drift, and safety routing drift across lifelong editing, MoE-LoRA tuning, and token-level routing.
Stabilization methods such as freeze-then-route, capability-aligned routing, and structured memory mechanisms are used to maintain robust system performance.

Searching arXiv for papers on routing drift and related MoE/continual learning formulations. Recent work suggests that Routing-Drift is best understood as an umbrella designation for failure modes in which the mapping from inputs, tokens, states, or requests to computational pathways changes under sequential updates, distribution shift, or hardware degradation, so that a system no longer activates the expert, adapter, attention pattern, threshold policy, or physical path that previously encoded the intended behavior. In lifelong model editing, this appears as semantic drift in routing memory (Luo et al., 11 Mar 2026). In continual MoE-LoRA tuning, it appears as Misaligned Co-drift, where router preferences and expert pathways drift together but not coherently (Hou et al., 19 Jan 2026). In safety-aligned MoE LLMs, it appears as safety routing drift on harmful prompts (Kim et al., 26 Sep 2025). In class-incremental and dynamic MoE systems, it appears when expert expansion perturbs old routing or when ambiguous tokens are pulled toward newly added experts (Guo et al., 17 May 2026, Zhao et al., 29 Mar 2026). Related formulations also arise in non-stationary forecasting, cloud-edge agent routing, continual vehicle-routing optimization, and quantum repeater networks, where temporal regime changes, network conditions, task drift, or calibration decay alter route selection and route quality (Zhu et al., 20 May 2026, Chen et al., 27 Nov 2025, Pei et al., 30 Jan 2026, Kumar, 21 Jun 2026).

1. Conceptual Scope and Taxonomy

Within continual learning and model editing, routing-drift denotes a change in which module is selected for an input that should have remained stably associated with an earlier edit, task, or capability. In SoLA, prior lifelong editing systems are described as vulnerable because routing structures such as clusters or learned routing codes are themselves updated during sequential editing; once those structures move, the same input may be matched to the wrong edit module, degrading both ERR and TRR through module mismatch (Luo et al., 11 Mar 2026). PASs-MoE sharpens this diagnosis by distinguishing router drift, where the gating distribution on old inputs changes across tasks, from expert drift, where the low-rank pathways of the selected experts themselves change; the coupled phenomenon is termed Misaligned Co-drift (Hou et al., 19 Jan 2026).

In expandable MoE systems for class-incremental learning, routing-drift is not merely a by-product of unfrozen expert weights. StaR-MoE argues that even when old experts remain frozen, introducing new experts re-normalizes the router over a larger pool, so old-class samples may acquire probability mass on newly added experts and traverse a different computation graph than the one that originally encoded their knowledge (Guo et al., 17 May 2026). LLaVA-DyMoE makes the same point at token granularity: ambiguous and old tokens in new-task data may be routed into new experts during training, and these misleading assignments later pull genuine old-task tokens toward the wrong expert group (Zhao et al., 29 Mar 2026).

In safety research, the term denotes drift in the routing-weight distribution that initially sent harmful prompts toward a small subset of safety-critical experts. SafeMoE defines this drift on harmful inputs as a divergence between the routing distribution of the initial safety-aligned model and that of the fine-tuned model, and argues that safety degradation in MoE LLMs is substantially a routing-policy failure rather than only a weight-space failure (Kim et al., 26 Sep 2025).

A broader, but structurally related, notion appears outside MoE. Dynamic TMoE identifies memoryless expert selection as unstable under abrupt regime shifts in non-stationary time series, producing erratic switching and poor continuity across adjacent patches (Zhu et al., 20 May 2026). In cloud-edge NetGPT, network-state changes move the optimal fallback boundary itself, so the routing threshold must adapt monotonically with bandwidth and round-trip time (Chen et al., 27 Nov 2025). In quantum repeater networks, the feasibility and quality of a route drift with calibration state and repeater heterogeneity, so path selection cannot be separated from hardware degradation and recalibration (Kumar, 21 Jun 2026). These cases suggest that routing-drift is not restricted to expert gating; it includes any instability in the rule that assigns work to pathways under evolving conditions.

2. Mechanisms That Produce Routing-Drift

A recurrent mechanism is joint but misaligned adaptation of router and experts. In MoE-LoRA continual instruction tuning, the update

$\Delta W(h)=\sum_{e=1}^{E}\pi_e(h)\,B_eA_e h$

depends simultaneously on the router weights $\pi_e(h)$ and the expert pathways $B_eA_e$ . PASs-MoE argues that conventional MoE-LoRA learns routing in a shared router space that is not anchored to the experts’ own functional subspaces, so the router can gradually stop selecting the expert whose low-rank pathway actually matches the input’s learned capability (Hou et al., 19 Jan 2026).

A second mechanism is state mutation in the routing memory. SoLA attributes failures in lifelong editing to continually updated clustering or routing structures. If semantic keys, cluster centers, or routing codes move after later edits, the semantic identity of earlier edits also moves, which changes inference-time retrieval even though the earlier edit module itself may not have been retrained (Luo et al., 11 Mar 2026). The paper treats this as especially damaging because routing is the gate that determines whether an edit is applied at all.

A third mechanism is expert-pool expansion with probability reallocation. StaR-MoE shows that expandable MoE introduces structural interference even with frozen old experts: as the router is expanded, old-class samples can be reassigned to future experts that were never trained on them (Guo et al., 17 May 2026). LLaVA-DyMoE adds a token-level explanation: old and ambiguous tokens in new-task data have little value for new-task learning but still contribute gradients that bias the router toward new experts, producing what the paper calls the token’s dilemma (Zhao et al., 29 Mar 2026).

A fourth mechanism is loss of context or temporal continuity in routing. Dynamic TMoE argues that memoryless gating treats each patch independently, which is inadequate when the best expert changes across time in a structured rather than random manner. The resulting temporal rigidity yields unstable expert assignment, especially when a series moves from stable seasonality to volatile trend (Zhu et al., 20 May 2026).

A fifth mechanism is score or calibration drift across routes. SafeMoE reports that routing changes for harmful prompts are concentrated especially in upper transformer layers, and that drift and harmfulness rise together during training (Kim et al., 26 Sep 2025). GR4CIL presents a related cross-task calibration problem in CLIP-based class-incremental learning: even when task-specific knowledge is organized separately, inconsistent modality gaps make cross-task scores poorly calibrated, which in turn makes reliable routing difficult (Wang et al., 20 Apr 2026).

A sixth mechanism is internal attention re-routing under context growth. The hallucination study on TruthfulQA argues that incremental context injection progressively redistributes attention, perturbs hidden-state geometry, and eventually produces an attention-locking regime in which hallucinations become resistant to correction (Wei et al., 22 May 2025). Although this work studies attention rather than external routers, it supplies a direct internal analogue of routing-drift: the model’s effective path through context changes until a new but erroneous routing pattern stabilizes.

3. Formalizations and Diagnostic Metrics

The literature formalizes routing-drift through several distinct but related objective functions and divergence measures. In SoLA, the routing problem is reduced to nearest-key retrieval over a fixed semantic table. With stored keys $K=\{k_1,\dots,k_N\}$ and query $q$ , inference chooses

$i=\arg\min_j \mathrm{dist}(q,k_j),$

and then applies the matched LoRA only if the nearest-key distance is below a threshold: $H_e = \begin{cases} H(W_0) & \text{if } d \ge a \ H(W_0, W_{Rm}) & \text{if } d < a, \end{cases}$ with $a=0.01$ in the paper (Luo et al., 11 Mar 2026). The point of the formulation is that neither the semantic key nor the LoRA module is updated after insertion.

PASs-MoE defines the expert’s Pathway Activation Subspace by the LoRA down-projection,

$\mathcal S_e=\mathrm{span}(A_e^\top),$

and uses the low-rank activation energy

$s_e(h)=\frac{1}{r}\lVert A_e h\rVert_2^2$

to derive routing weights. Router drift is then studied empirically by comparing gating distributions $\pi_e(h)$ 0 for old inputs at different checkpoints, using distributional divergence such as Jensen–Shannon divergence (Hou et al., 19 Jan 2026). The same framework defines rank-direction importance and a stabilization loss so that directions important to previous tasks are constrained more strongly than unimportant ones.

SafeMoE gives perhaps the most explicit drift metric. For a harmful input $\pi_e(h)$ 1, safety routing drift is

$\pi_e(h)$ 2

where $\pi_e(h)$ 3 is the routing-weight distribution over experts (Kim et al., 26 Sep 2025). The regularizer extends this to multiple layers and harmful samples, using a temperature $\pi_e(h)$ 4 so that smaller $\pi_e(h)$ 5 concentrates the penalty more heavily on top-ranked experts.

StaR-MoE formulates old-class routing preservation through class-wise dense routing anchors. Historical distributions are zero-padded so that classes learned earlier are assigned zero mass on experts added later, and the alignment loss is

$\pi_e(h)$ 6

The paper further proves a routing-drift bound: $\pi_e(h)$ 7 linking the optimization objective directly to bounded total drift (Guo et al., 17 May 2026).

LLaVA-DyMoE introduces a token-level ambiguity score using the maximum old-group and new-group router logits,

$\pi_e(h)$ 8

and declares a token ambiguous when $\pi_e(h)$ 9 (Zhao et al., 29 Mar 2026). This converts routing-drift analysis from sample level to token level and enables selective masking of tokens that should not update new experts.

Dynamic TMoE formalizes drift at the data-distribution level rather than the expert-assignment level. It uses a GRU-based hidden state

$B_eA_e$ 0

for temporally consistent routing and declares distribution shift when the squared MMD between reference and current windows exceeds an adaptive threshold

$B_eA_e$ 1

with $B_eA_e$ 2 in the implementation details (Zhu et al., 20 May 2026).

The hallucination literature contributes internal drift metrics. The hidden-state shift is measured by cosine drift, while attention reconfiguration is measured by Jensen–Shannon divergence and rank-based Spearman drift. The reported convergence of JS-Drift to about $B_eA_e$ 3 together with Spearman-Drift approaching $B_eA_e$ 4 is interpreted as an attention-locking threshold (Wei et al., 22 May 2025). This is a rare case where a routing-like phase transition is diagnosed directly inside the model.

4. Stabilization and Alignment Strategies

One major design line is freeze-then-route. SoLA allocates one LoRA per edit, freezes the adapter after training, extracts a semantic key from the input representation, and adds the pair to a fixed routing memory. Because both the key and the adapter are frozen, later edits do not alter earlier routing decisions. The same design also yields reversible rollback: deleting the key prevents activation of the corresponding edit module and restores base-model behavior without retraining (Luo et al., 11 Mar 2026).

A second line is capability-aligned routing. PASs-MoE replaces an independently learned router with PAS-guided routing grounded in the expert’s own low-rank response. Its PAS-guided Reweighting computes routing from the activation energy $B_eA_e$ 5, and PAS-aware Rank Stabilization selectively constrains historically important rank directions rather than freezing the entire expert (Hou et al., 19 Jan 2026). The central principle is that routing and preservation should share the same coordinate system.

A third line is historical routing preservation under expansion. StaR-MoE stores class-wise routing anchors and aligns current dense routing to these historical targets using synthetic router inputs drawn from class-wise Gaussian statistics. Its complementary Asymmetric Capacity Regularization penalizes only overloaded experts, thereby encouraging use of new capacity without forcing uniform expert utilization that would destroy specialization (Guo et al., 17 May 2026).

A fourth line is token-selective routing control. LLaVA-DyMoE introduces Token Assignment Guidance, which allows only tokens that are clearly new-dominant and unambiguous to train the new expert group. It complements this with Routing Score Regularization, consisting of an exclusivity loss that discourages simultaneous activation of old and new expert groups and a specialization loss that encourages new experts to specialize on genuinely new-task tokens (Zhao et al., 29 Mar 2026).

A fifth line is routing-distribution regularization for safety. SafeMoE adds a KL penalty between the routing distributions of the aligned and fine-tuned models on harmful instructions. The paper uses a bi-level greedy schedule in which ordinary fine-tuning steps are interleaved with periodic routing-regularization steps on harmful samples, while aligned-model routing weights are precomputed to reduce overhead (Kim et al., 26 Sep 2025).

A sixth line is history-aware or structured routing under non-stationarity. Dynamic TMoE couples MMD-triggered expert instantiation and pruning with a Temporal Memory Router that uses recurrent state and an Anomaly State Repository to retrieve relevant past shift states, thereby stabilizing routing without test-time updates (Zhu et al., 20 May 2026). GatedLinear pursues a lighter-weight alternative: it routes among three complementary linear bases by a Tri-Factorized Fusion Gate with channel-specific preferences, horizon-aware offsets, and phase-indexed biases from future time marks, allowing point-wise soft routing across trend-seasonal continuation, difference-based drift, and phase-aligned recurrence (Tan et al., 10 Jul 2026).

A seventh line is score-calibrated routing across task branches. GR4CIL reduces old-knowledge drift by using task-specific visual LoRA modules and a shared textual semantic space stabilized by anchor and separation losses. It then adds an orthogonal compensation mechanism in the complement of the text subspace, enlarging the margin between the ground-truth task and competing tasks and making task-aware score competition more reliable (Wang et al., 20 Apr 2026).

In systems settings, stabilization takes yet other forms. NetGPT derives a unique optimal score threshold for cloud fallback and adapts it monotonically with network state, while a schema-preserving RL objective combines a reverse-KL trust-region step and a forward-KL realignment toward the SFT prior to constrain policy drift (Chen et al., 27 Nov 2025). DREE, in lifelong vehicle-routing under continually drifting tasks, addresses drift through replay of both problem instances and behaviors, with Experience Enhancement updating buffered trajectories when better solutions are found later (Pei et al., 30 Jan 2026). In quantum repeater networks, calibration-aware orchestration allocates activation and calibration phases so that routing respects fidelity decay and shared-link constraints rather than treating drift as an external nuisance (Kumar, 21 Jun 2026).

5. Empirical Record Across Domains

The empirical literature is unusually consistent in showing that routing stability is not an auxiliary refinement but a primary determinant of retention, safety, and adaptation quality.

Domain	Representative empirical result	Paper
Continual instruction tuning with MoE-LoRA	AP = 48.46, BWT = -2.15; MoELoRA (Softmax) has AP = 43.36, BWT = -6.64	(Hou et al., 19 Jan 2026)
Non-stationary time series forecasting	Average reductions of 10.4% MSE and 7.8% MAE over competitive baselines across nine benchmarks	(Zhu et al., 20 May 2026)
Time series forecasting with structured soft routing	Best average MSE = 0.260 and MAE = 0.298; best on 11 of 16 dataset-level metrics	(Tan et al., 10 Jul 2026)
MoE safety under harmful fine-tuning	On OLMoE SAMSum, harmfulness score 62.0 $B_eA_e$ 6 5.0; LoRA fine-tuning overhead about 2.13%	(Kim et al., 26 Sep 2025)
Expandable MoE class-incremental learning	CIFAR-100: $B_eA_e$ 7, $B_eA_e$ 8; ImageNet-A: $B_eA_e$ 9, $K=\{k_1,\dots,k_N\}$ 0	(Guo et al., 17 May 2026)
Dynamic MoE for LVLM continual tuning	MFN 57.03 vs. 49.68, MAA 57.70 vs. 49.50, BWT -4.67 vs. -16.67 against IncMoELoRA	(Zhao et al., 29 Mar 2026)
CLIP-based class-incremental routing	89.35/83.22 on CIFAR-100 and 89.50/84.15 on ImageNet-R; average margin improvement around 20.34%	(Wang et al., 20 Apr 2026)

Additional evidence comes from internal-state analysis. In the hallucination study, hallucination rates and internal drift both grow monotonically under incremental context injection and often plateau after roughly 5–7 rounds; JS-Drift saturates near 0.69 and Spearman-Drift approaches 0 around 6–8 rounds, marking the reported attention-locking regime (Wei et al., 22 May 2025). In cloud-edge NetGPT, the empirically optimal fallback threshold decreases from approximately $K=\{k_1,\dots,k_N\}$ 1 to $K=\{k_1,\dots,k_N\}$ 2 to $K=\{k_1,\dots,k_N\}$ 3, matching the theory that worse network conditions should make the router more permissive to edge acceptance and less eager to offload (Chen et al., 27 Nov 2025).

The empirical record also shows that different stabilization mechanisms isolate different failure sources. Dynamic TMoE reports that replacing the GRU router with Linear or MLP routers increases MSE and MAE, and that removing anomaly memory degrades performance, supporting the claim that routing continuity matters in non-stationary forecasting (Zhu et al., 20 May 2026). GatedLinear reports that replacing its tri-factorized gate with a plain shared softmax produces the largest ablation loss among the tested routing components, supporting the claim that routing must be structured by channel, horizon, and phase rather than shared globally (Tan et al., 10 Jul 2026). Across these domains, the evidence repeatedly links lower routing drift to better retention or better adaptation.

6. Limitations, Misconceptions, and Open Problems

A common misconception is that freezing experts is sufficient to prevent forgetting. The literature rejects this. StaR-MoE shows that old experts may remain frozen while old samples are nonetheless redirected to new experts through router expansion (Guo et al., 17 May 2026). LLaVA-DyMoE reaches the same conclusion at token level: forgetting can arise because the router learns the wrong assignment policy even though older expert parameters are untouched (Zhao et al., 29 Mar 2026).

A second misconception is that routing is only an implementation detail. SoLA explicitly argues that in lifelong editing routing determines whether an edit is actually applied, and therefore whether edited knowledge can be retrieved or revoked (Luo et al., 11 Mar 2026). SafeMoE makes the analogous argument for MoE safety: harmful inputs are often safe because they are routed to safety-critical experts, so preserving the routing policy is part of preserving the safety mechanism itself (Kim et al., 26 Sep 2025).

A third misconception is that all uses of “drift” in routing research refer to the same object. They do not. In continual learning and MoE work, routing-drift usually denotes assignment drift or gating drift. In stochastic network optimization, by contrast, drift-plus-penalty refers to Lyapunov drift of queue backlogs, not to a changing expert-assignment policy. DPOP, for example, proves $K=\{k_1,\dots,k_N\}$ 4 regret for joint routing and scheduling with unknown costs by combining Lyapunov control with optimistic cost estimates, but this is conceptually distinct from MoE routing-drift (Chadaga et al., 3 Sep 2025). The later neural drift-plus-penalty work similarly “neuralizes” backlog design while preserving a bounded-backlog stability guarantee, again using drift in the Lyapunov rather than expert-selection sense (Rashwan et al., 11 Sep 2025).

The current mitigation methods also leave unresolved technical dependencies. SoLA’s stability depends on the quality of semantic embeddings for query-key matching, on the assumption that a reliable key can be derived from the last-token hidden state, and on threshold tuning for the master decision layer (Luo et al., 11 Mar 2026). Dynamic TMoE presumes that MMD over sliding windows is an adequate trigger for architectural evolution and that past anomaly states can usefully summarize recurrent shifts (Zhu et al., 20 May 2026). GR4CIL shows that cross-task routing reliability is entangled with modality-gap calibration, implying that routing stability can fail even when task-specific knowledge is otherwise isolated (Wang et al., 20 Apr 2026). In quantum networking, routing quality depends on calibration schedules and end-to-end fidelity estimates, so path choice cannot be separated from physical maintenance and observability (Kumar, 21 Jun 2026).

The internal-attention literature raises a further challenge: once a system enters an attention-locking regime, late correction may have reduced effect because the erroneous routing pattern has already stabilized (Wei et al., 22 May 2025). This suggests that prevention may be easier than repair. A plausible implication is that future work will increasingly treat routing stability as a first-class systems property spanning semantic memory, expert gating, score calibration, temporal continuity, and hardware health, rather than as a narrow architectural choice.