Online Mixture-of-Experts (OMoE): Adaptive Routing

• Online Mixture-of-Experts (OMoE) is a framework that sequentially adapts expert selection, gating, and parameters to handle streaming and non-stationary data.
• It integrates methods like sparse gating, continual adaptation, and mirror descent to balance computational efficiency with expertise specialization.
• OMoE enhances performance in diverse applications—from speech conversion to time-series forecasting—by dynamically controlling resource allocation and expert routing.

Searching arXiv for papers and terminology usage around “Online Mixture-of-Experts (OMoE)”. Online Mixture-of-Experts (OMoE) denotes a family of mixture-of-experts formulations in which expert selection, expert weighting, expert parameters, or routing-control variables are updated under sequential, streaming, test-time, or online-computation constraints rather than being treated as purely static after offline fitting. In current arXiv usage, the term is not uniform: it can refer to streaming concept-drift adaptation with an online router and incremental experts, online balancing of routing loads by mirror descent, online test-time rerouting of pretrained sparse models, dynamic conditional-density MoE inference for time-varying data, or orthogonalized expert optimization with an alternating “online-like” update schedule (Aspis et al., 24 Jul 2025, Chen et al., 14 May 2026, Su et al., 16 Oct 2025, Munezero et al., 2021, Liu et al., 2023). This suggests that OMoE is best understood as a design space organized around sequential adaptation and conditional computation, rather than as a single canonical algorithm.

1. Terminological scope and historical usage

The literature assigns the label “online” to several distinct MoE regimes. In one line of work, “online” means inference-time efficiency: MoEVC inserts sparse gating around convolutional layers in voice conversion so that channels with zero gates are not computed, reducing online computation while preserving or improving conversion quality (Chang et al., 2019). In a second line, “online” means continual adaptation to non-stationary streams: DriftMoE processes instances one-by-one under the interleaved test-then-train protocol and co-trains a neural router with incremental Hoeffding tree experts (Aspis et al., 24 Jul 2025). In a third line, “online” refers to stochastic optimization of routing balance over mini-batches by EMA and mirror descent, as in $\phi$-balancing (Chen et al., 14 May 2026). In a fourth, it denotes test-time adaptation of routing decisions during generation, without external data, by repeatedly optimizing router-logit biases from the current context (Su et al., 16 Oct 2025).

OMoE interpretation	Core mechanism	Reference
Online computation efficiency	Sparse channel gating and skipped convolutions	(Chang et al., 2019)
Streaming concept-drift adaptation	Neural router + incremental Hoeffding tree experts	(Aspis et al., 24 Jul 2025)
Dynamic online prediction	Time-varying gates and experts with sequential Monte Carlo	(Munezero et al., 2021)
Online/offline non-stationary time series	Meta-expert combining static offline experts and one online expert	(Zhao et al., 2022)
Population-level online balancing	EMA of routing probabilities + mirror-descent dual updates	(Chen et al., 14 May 2026)
Online test-time rerouting	Additive router-logit vectors updated during generation	(Su et al., 16 Oct 2025)

The same term is also used in optimization-centric rather than stream-centric senses. “OMoE” in the orthogonal-expert literature denotes an orthogonal expert optimizer that updates experts in directions projected against the subspaces of other experts, using alternating regular and orthogonal steps (Liu et al., 2023). A related PEFT formulation, also named OMoE, enforces orthogonality among LoRA experts by Gram–Schmidt projection onto the Stiefel manifold (Feng et al., 17 Jan 2025). The coexistence of these meanings is a recurring source of confusion in the literature.

2. Shared formalism and architectural primitives

Despite the terminological spread, most OMoE formulations retain the classical MoE decomposition

$MoE(x) = \sum_{i=1}^{k} g_i(x)\cdot f_i(x),$

with expert functions $f_i$ and an input-dependent gate $g_i(x)$ (Liu et al., 2023). The distinctive feature is that some part of this decomposition is updated or interpreted sequentially.

In DriftMoE, the router is an MLP $R_\theta$ that outputs logits $\mathbf{o}_t$ and softmax weights $\mathbf{w}_t$ for expert choice,

$$\mathbf{o}_t = R_\theta(x_t), \qquad \mathbf{w}_t = \text{softmax}(\mathbf{o}_t),$$

while the experts are incremental Hoeffding trees that emit class-probability vectors (Aspis et al., 24 Jul 2025). Prediction uses hard routing to the single most probable expert, but training uses Top-$k$ routing in the multi-class variant. By contrast, the dynamic MoE model for online prediction is a conditional-density model

$$f_j(y_j \mid \tilde{\mathbf x}_j, \boldsymbol{\omega}_j, \boldsymbol{\lambda}_j) = \sum_{k=1}^K \omega_{jk}(\mathbf z_j)\, f_{jk}\!\left(y_j \mid \lambda_{jk}(\mathbf x_j)\right),$$

with both gating and expert parameters evolving in time by random walks, and filtering performed sequentially by SMC (Munezero et al., 2021).

The same general pattern appears in non-stationary time-series prediction with a different division of labor. The Mixture of Online and Offline Experts maintains $MoE(x) = \sum_{i=1}^{k} g_i(x)\cdot f_i(x),$0 static offline experts learned from previous intervals and a single dynamic online expert for the current interval, while a meta-expert forms the aggregate predictor

$MoE(x) = \sum_{i=1}^{k} g_i(x)\cdot f_i(x),$1

and updates the mixture weights by exponential weighting (Zhao et al., 2022). In MEC, the abstraction shifts from model components to systems components: each server is an expert, the gating network scores a newly arrived task, and routing is constrained by server availability and delay (Li et al., 2024).

A different but structurally related block-online perspective appears in distributed MoE learning. There, local MoEs are trained on distributed subsets, then reduced into a single $MoE(x) = \sum_{i=1}^{k} g_i(x)\cdot f_i(x),$2-expert MoE by minimizing an expected transportation divergence and solving the resulting reduction by MM (Chamroukhi et al., 2023). The paper is not a sample-level online algorithm, but its aggregation operator can be interpreted as a model-level online update over arriving data blocks. This suggests a useful distinction between sample-online OMoE and block-online OMoE.

3. Streaming, sequential, and continual-learning regimes

The clearest stream-native OMoE formulation is DriftMoE. Data arrive as an infinite sequence $MoE(x) = \sum_{i=1}^{k} g_i(x)\cdot f_i(x),$3 under abrupt, gradual, continuous, or real-world drift, and instances are processed one-by-one with interleaved test-then-train. Experts are updated per instance, while the router is updated online but in mini-batches of recent instances. The core supervision signal is a multi-hot correctness mask

$MoE(x) = \sum_{i=1}^{k} g_i(x)\cdot f_i(x),$4

and the router is trained by a per-expert BCE loss over buffered logits and masks (Aspis et al., 24 Jul 2025). The paper evaluates nine benchmarks and reports, for example, that on Airlines, MoE-Data reaches $MoE(x) = \sum_{i=1}^{k} g_i(x)\cdot f_i(x),$5, exceeding the best baseline SRP at $MoE(x) = \sum_{i=1}^{k} g_i(x)\cdot f_i(x),$6 (Aspis et al., 24 Jul 2025). MoE-Data is described as more stable across LED/SEA/Airlines, whereas MoE-Task is more reactive and stronger on fast RBF drift but may collapse on class-imbalanced streams such as Electricity and CoverType (Aspis et al., 24 Jul 2025).

The dynamic conditional-density formulation addresses online prediction from a Bayesian state-space perspective. Expert coefficients and gating coefficients evolve as Gaussian random walks,

$MoE(x) = \sum_{i=1}^{k} g_i(x)\cdot f_i(x),$7

and filtering is performed with a tailored marginal particle filter that uses linear-Bayes and EM-style structure to build a proposal distribution (Munezero et al., 2021). The model accommodates static parameters as a special case and was evaluated on simulated settings and on industrial software-fault prediction, where dynamic two-component Poisson MoE models outperform static counterparts in log predictive score (Munezero et al., 2021).

The online/offline time-series formulation separates historical transfer from current adaptation. Offline intervals each contain $MoE(x) = \sum_{i=1}^{k} g_i(x)\cdot f_i(x),$8 samples, the current online interval contains $MoE(x) = \sum_{i=1}^{k} g_i(x)\cdot f_i(x),$9 observed sequentially, and the meta-expert updates

$f_i$0

to combine experts according to recent losses (Zhao et al., 2022). The paper derives an $f_i$1 regret bound and an excess-risk bound for the averaged meta-solution, thereby formalizing a hybrid OMoE regime in which some experts are frozen and one remains fully online (Zhao et al., 2022).

A systems-theoretic continual-learning version appears in MEC. Here, time is slotted, each arriving mobile-user task must be assigned immediately, and servers can be unavailable while busy. The router selects

$f_i$2

so routing depends both on task features and current availability (Li et al., 2024). The theory derives a minimum number of experts required so that each task cluster can match to a specialized available expert with high probability, and proves that naive MEC offloading yields growing generalization error whereas MoE routing converges to cluster-specialized experts (Li et al., 2024). The same analysis also shows that once the number of experts is sufficient, adding more experts delays convergence and worsens finite-horizon generalization error (Li et al., 2024).

Bandit-style OMoE reframes the problem as online collective decision-making. Given a context and a committee of experts, the learner must adaptively select a subset or a weight vector to optimize aggregate accuracy. The paper proposes a UCB-based successive elimination algorithm for egalitarian committees and an online weighted-majority-voting mechanism for weighted committees, with regret measured against the optimal committee or weight vector rather than the best single expert (Liu et al., 19 Oct 2025). This is a decision-level OMoE rather than a differentiable internal architecture.

4. Routing, balancing, and conditional computation

A major OMoE theme is that online adaptation often acts primarily through routing, gating, or sparsity rather than through full retraining of all expert parameters. In MoEVC, the gate is a layer-wise channel selector,

$f_i$3

and gated convolution is written as

$f_i$4

Because ReLU and an $f_i$5 penalty drive many entries of $f_i$6 to zero, channels with zero gate values are not computed at inference time (Chang et al., 2019). The paper reports FLOPs Reduction Rate values of $f_i$7, $f_i$8, $f_i$9, $g_i(x)$0, $g_i(x)$1, $g_i(x)$2, and $g_i(x)$3, and states that the system can reduce more than $g_i(x)$4 of FLOPs without harming and even increasing the quality of converted speech (Chang et al., 2019). At the operating point denoted B, FRR is approximately $g_i(x)$5, MOSNet is approximately $g_i(x)$6, naturalness MOS averages $g_i(x)$7 versus $g_i(x)$8 for baseline ACVAE, and similarity averages $g_i(x)$9 versus $R_\theta$0 (Chang et al., 2019). This is an online-computation interpretation of OMoE in which expert execution itself becomes conditional.

In DriftMoE, routing is supervised by expert correctness rather than by the downstream class label directly. The router is updated from a BCE loss over the multi-hot mask, while experts are updated either on Top-$R_\theta$1 routed data in the multi-class regime or via one-vs-rest tasks in the task-based regime (Aspis et al., 24 Jul 2025). This symbiotic loop is central: experts teach the router which experts were correct, and the router decides which experts receive future examples. The paper fixes $R_\theta$2 and Top-$R_\theta$3 for MoE-Data after a grid search showing a plateau for $R_\theta$4 and $R_\theta$5, retaining approximately $R_\theta$6 of peak accuracy while saving approximately $R_\theta$7 compute versus a 20-expert configuration (Aspis et al., 24 Jul 2025).

Population-level routing balance is treated formally in $R_\theta$8-balancing. Let

$R_\theta$9

be the expected routing distribution. The balancing term is

$\mathbf{o}_t$0

where $\mathbf{o}_t$1 is strictly convex, symmetric, and differentiable (Chen et al., 14 May 2026). By convex duality and mirror descent, the dual update becomes exactly the EMA rule

$\mathbf{o}_t$2

and the per-step auxiliary loss becomes

$\mathbf{o}_t$3

The paper reports that negative Shannon entropy gives the best balance with $\mathbf{o}_t$4, and that $\mathbf{o}_t$5 performs best, whereas $\mathbf{o}_t$6 close to $\mathbf{o}_t$7 behaves like per-batch statistics and becomes unstable (Chen et al., 14 May 2026). Unlike Switch-style balancing, the objective is explicitly population-level rather than mini-batch-level (Chen et al., 14 May 2026).

These three cases illustrate distinct online-control roles for gating: sparse execution control, streaming specialization control, and population-level load control. This suggests that routing in OMoE is not merely an assignment mechanism; it is often the locus of the online algorithm itself.

5. Expert diversity, orthogonality, and test-time adaptation

Another major OMoE current addresses the failure mode of homogeneous experts. In the orthogonal-expert optimizer for LLMs, standard AdamW fine-tuning can leave experts almost indistinguishable: the authors report that more than $\mathbf{o}_t$8 of parameters are “similar” across experts in a well-performed MoE model (Liu et al., 2023). Their OMoE adapts Orthogonal Weight Modification to MoE by alternating ordinary updates and orthogonalized updates, using per-expert projectors and the average projector of the other experts,

$\mathbf{o}_t$9

The procedure improves GLUE average scores from $\mathbf{w}_t$0 to $\mathbf{w}_t$1 for BERT-MoE, from $\mathbf{w}_t$2 to $\mathbf{w}_t$3 for RoBERTa-MoE, and from $\mathbf{w}_t$4 to $\mathbf{w}_t$5 for ALBERT-MoE (Liu et al., 2023). It also improves SQuAD1.1 F1 from $\mathbf{w}_t$6 to $\mathbf{w}_t$7 for BERT-MoE and from $\mathbf{w}_t$8 to $\mathbf{w}_t$9 for RoBERTa-MoE, while increasing optimizer memory to $$\mathbf{o}_t = R_\theta(x_t), \qquad \mathbf{w}_t = \text{softmax}(\mathbf{o}_t),$$0 the standard MoE optimizer and adding $$\mathbf{o}_t = R_\theta(x_t), \qquad \mathbf{w}_t = \text{softmax}(\mathbf{o}_t),$$1 GMacs per O-step (Liu et al., 2023). Here “online” refers to recursive projector updates over mini-batches and alternating update phases rather than to streaming deployment.

The orthogonal LoRA variant pushes the same concern into PEFT. OMoE replaces vanilla MoE-LoRA aggregation with Gram–Schmidt-orthogonalized expert outputs,

$$\mathbf{o}_t = R_\theta(x_t), \qquad \mathbf{w}_t = \text{softmax}(\mathbf{o}_t),$$2

thereby enforcing $$\mathbf{o}_t = R_\theta(x_t), \qquad \mathbf{w}_t = \text{softmax}(\mathbf{o}_t),$$3 on the Stiefel manifold at the representation level (Feng et al., 17 Jan 2025). In the main setup, only $$\mathbf{o}_t = R_\theta(x_t), \qquad \mathbf{w}_t = \text{softmax}(\mathbf{o}_t),$$4 experts are used per LoRA site. On LLaMA-2 7B, tunable parameters drop to $$\mathbf{o}_t = R_\theta(x_t), \qquad \mathbf{w}_t = \text{softmax}(\mathbf{o}_t),$$5 for OMoE-LoRA / OMoE-DoRA versus $$\mathbf{o}_t = R_\theta(x_t), \qquad \mathbf{w}_t = \text{softmax}(\mathbf{o}_t),$$6 for LoRA / DoRA / MixLoRA, and OMoE (Soft) reports $$\mathbf{o}_t = R_\theta(x_t), \qquad \mathbf{w}_t = \text{softmax}(\mathbf{o}_t),$$7 ms latency with $$\mathbf{o}_t = R_\theta(x_t), \qquad \mathbf{w}_t = \text{softmax}(\mathbf{o}_t),$$8 MiB extra memory versus MixLoRA’s $$\mathbf{o}_t = R_\theta(x_t), \qquad \mathbf{w}_t = \text{softmax}(\mathbf{o}_t),$$9 ms and $k$0 MiB (Feng et al., 17 Jan 2025). The paper argues that fewer but more diverse experts are preferable to larger redundant expert sets (Feng et al., 17 Jan 2025).

Test-time adaptation introduces a different notion of online expert management. “Rewiring Experts on the Fly” keeps the pretrained experts and base router frozen, but adds per-layer additive bias vectors to router logits,

$k$1

with $k$2 (Su et al., 16 Oct 2025). During prefill and then every $k$3 generated tokens by default, the method runs $k$4 Adam steps with learning rate $k$5, weight decay $k$6, and $k$7 to minimize the language-model loss on the current context (Su et al., 16 Oct 2025). The paper reports a $k$8 improvement on HumanEval with OLMoE and $k$9 average gains when combined with self-consistency on DeepSeek-V2-Lite (Su et al., 16 Oct 2025). It also reports that updating every $$f_j(y_j \mid \tilde{\mathbf x}_j, \boldsymbol{\omega}_j, \boldsymbol{\lambda}_j) = \sum_{k=1}^K \omega_{jk}(\mathbf z_j)\, f_{jk}\!\left(y_j \mid \lambda_{jk}(\mathbf x_j)\right),$$0 tokens works best, while updating all layers or adapting too frequently degrades performance (Su et al., 16 Oct 2025). This is a fully test-time, data-free OMoE: expert routing is optimized while the model is running.

6. Empirical landscape, misconceptions, and open questions

OMoE systems have been evaluated in speech conversion, non-stationary classification streams, probabilistic time-series prediction, edge computing, LLM fine-tuning, test-time reasoning, and expert committee aggregation (Chang et al., 2019, Aspis et al., 24 Jul 2025, Munezero et al., 2021, Li et al., 2024, Liu et al., 2023, Su et al., 16 Oct 2025, Liu et al., 19 Oct 2025). Across these domains, two empirical regularities recur. First, moderate specialization or sparsity often improves both efficiency and accuracy: MoEVC improves naturalness and similarity at about $$f_j(y_j \mid \tilde{\mathbf x}_j, \boldsymbol{\omega}_j, \boldsymbol{\lambda}_j) = \sum_{k=1}^K \omega_{jk}(\mathbf z_j)\, f_{jk}\!\left(y_j \mid \lambda_{jk}(\mathbf x_j)\right),$$1 FRR rather than at either very low or very high sparsity (Chang et al., 2019); DriftMoE matches or exceeds heavyweight adaptive ensembles on several drift benchmarks using 12 experts rather than 100 trees (Aspis et al., 24 Jul 2025); orthogonal optimization improves downstream performance precisely by making experts less redundant (Liu et al., 2023). Second, online adaptation is most effective when the updated object is small and structured: router logits, gating EMAs, mixture weights, orthogonal projectors, or a single online expert.

A common misconception is that OMoE always means “MoE trained on a stream.” The literature does not support that restriction. Some papers use the label for online balancing of expected routing loads (Chen et al., 14 May 2026), some for dynamic test-time rerouting without parameter updates to experts (Su et al., 16 Oct 2025), some for online committee selection in bandit settings (Liu et al., 19 Oct 2025), and some for orthogonal mini-batch training schedules (Liu et al., 2023). Conversely, some work that is clearly relevant to online MoE behavior, such as MoEVC’s conditional skipping of convolutional channels, does not use the acronym OMoE in its title but operationalizes online expert selection in a resource-constrained setting (Chang et al., 2019). This suggests that the defining criterion is online adaptation of expert usage or expert coordination, not a specific model family.

The main unresolved issues are also consistent across subfields. MoEVC reports FLOPs reduction but not explicit latency or real-time factor, and it leaves expert interpretability largely open (Chang et al., 2019). DriftMoE does not explicitly model recurring concepts, performs poorly on highly imbalanced streams such as Electricity and CoverType, and identifies dynamic expert allocation and uncertainty-aware routing as future work (Aspis et al., 24 Jul 2025). $$f_j(y_j \mid \tilde{\mathbf x}_j, \boldsymbol{\omega}_j, \boldsymbol{\lambda}_j) = \sum_{k=1}^K \omega_{jk}(\mathbf z_j)\, f_{jk}\!\left(y_j \mid \lambda_{jk}(\mathbf x_j)\right),$$2-balancing derives the mirror-descent equivalence but does not provide explicit regret bounds or finite-time imbalance guarantees (Chen et al., 14 May 2026). Orthogonal methods improve diversity but can degrade when projector estimation is poor, especially on low-resource tasks or with too many experts (Liu et al., 2023, Feng et al., 17 Jan 2025). Test-time rerouting remains substantially more expensive than vanilla inference, at approximately $$f_j(y_j \mid \tilde{\mathbf x}_j, \boldsymbol{\omega}_j, \boldsymbol{\lambda}_j) = \sum_{k=1}^K \omega_{jk}(\mathbf z_j)\, f_{jk}\!\left(y_j \mid \lambda_{jk}(\mathbf x_j)\right),$$3 the FLOPs of the baseline on one reported HumanEval configuration, and can over-adapt if the update interval is too short (Su et al., 16 Oct 2025). The bandit OMoE framework yields no-regret guarantees, but weighted-majority optimization relies on a mixed-integer program whose direct form scales poorly with expert count (Liu et al., 19 Oct 2025).

The broad research direction is therefore not merely to enlarge expert pools, but to develop online control laws for when experts are created, activated, balanced, orthogonalized, merged, or rerouted. The current literature shows that OMoE can mean sparse conditional execution, continual routing under drift, online convex balancing, online collective decision-making, or test-time pathway editing. What unifies these strands is the attempt to make expert specialization usable under sequential constraints, while keeping the active computation or active coordination set much smaller than the nominal model capacity.