Multi-Stage Mixture of Movement Experts

• MoME is a hierarchical modular architecture that divides movement skill learning into specialized, independently optimized expert modules.
• The framework employs staged routing and reward augmentation to avoid mode averaging and enhance precision in tasks like robotic manipulation and trajectory prediction.
• Iterative expert addition and adaptive gating enable scalable, context-sensitive fusion, yielding diverse experts for applications from robotics to human motion analysis.

A Multi-Stage Mixture of Movement Experts (MoME) is a hierarchical modular architecture for movement skill learning, decision-making, or prediction, in which multiple expert policies (or models) are organized so that each specializes in a subregion of task context, trajectory structure, or behavioral mode. These experts are integrated through staged routing, adaptive gating, and optimization strategies, enabling high versatility, precision, and scalability in complex movement-related tasks. MoME frameworks have been applied in diverse domains including robotics, trajectory forecasting, human mobility modeling, and behavioral inference.

1. Conceptual Foundations and Objective Decomposition

The MoME paradigm extends classic Mixture of Experts (MoE) models by structuring movement skill learning into multiple stages, each refining the specialization and integration of experts. The foundational approach formalizes the overall objective as a maximum entropy reinforcement learning (RL) problem, in which the aim is to maximize expected reward subject to an entropy or diversity bonus:

$$\max_{\pi(\theta|c),\,\pi(c)}~\mathbb{E}_{c\sim p(c)}\Big[\mathbb{E}_{\theta\sim\pi(\theta|c)}[R(\theta,c)] + \alpha H[\pi(\theta|c)]\Big] - \beta\,\mathrm{KL}\big(\pi(c)\|\ p(c)\big).$$

Here, the policy $\pi(\theta|c)$ is parameterized as a mixture $\pi(\theta|c) = \sum_o \pi(o|c)\pi(\theta|o,c)$. In MoME, each mixture component (expert) is associated with a local context distribution $\pi(c|o)$, and the objective is decomposed by introducing variational lower bounds. This decomposition enables optimization of each expert and its local context distribution independently by solving an expert-specific RL problem, as follows:

• Each expert module $\pi(\theta|o,c)$ is updated via maximization of a local augmented reward.
• Expert-specific context distributions $\pi(c|o)$ are optimized to concentrate learning on tractable subregions.
• The gating weights $\pi(o)$ are adapted to maintain diversity and balance.

Such a decomposition allows staged, modular addition of experts and isolation of learning signals, paving the way for curriculum learning and local specialization (Celik et al., 2021).

2. Expert Specialization and Local Context Curriculum

A defining feature of MoME is local specialization. Each expert, instead of representing the entire range of possible movements or contexts, focuses on a partition of the task space, as encoded by $\pi(c|o)$. This local focus avoids the mode-averaging problem and enables high-precision, high-quality movement primitives. Local context specialization is dynamically achieved through reward augmentation terms that penalize overlap between experts’ coverage, thus incentivizing diversity and accuracy among experts. The curriculum emerges naturally: early experts solve the simplest and most common subproblems, and further components are introduced to capture underrepresented or challenging modes.

This approach has been successfully demonstrated in robotic manipulation and sports tasks (e.g., planar reaching, Beer Pong, Table Tennis), where the learned experts not only achieve higher mean rewards but also generate greater behavioral diversity and more precise context coverage compared to single-policy or standard hierarchical skill search baselines, such as HiREPS or LaDiPS (Celik et al., 2021).

3. Iterative and Hierarchical Expert Expansion

MoME architectures use an iterative component addition strategy: initial training starts with a single expert, and new experts are introduced sequentially. At each stage:

• All previously trained experts are frozen.
• A new expert and its local context curriculum are randomly initialized.
• The new expert is trained for a fixed number of iterations using context samples generated from its dedicated local context distribution.

The reward design ensures that the newly added expert targets regions of context not yet adequately represented by the existing mixture, maximizing complementary skill acquisition. This modular, staged expansion results in a library of specialized experts—scalable and adaptable, as more are added to increase repertoire diversity or fill performance gaps.

4. Comparative Performance and Evaluation

Empirical evaluations across multiple challenging robotic skill tasks and trajectory prediction scenarios confirm the superiority of the MoME approach relative to classical alternatives:

Task	MoME Success Rate (%)	Baseline (HiREPS) Success Rate (%)	Notes
Beer Pong (context: 2D cup)	~92	~75	Diverse throwing skills, full context space coverage
Table Tennis (context: 4D)	Higher (see paper)	Lower	Experts capture forehand/backhand, high task reward, diverse strikes
Planar Reaching	High	Few active experts	MoME: many distinct modes, HiREPS: only 3/60 experts active

Ablation studies confirm that omitting expert-specialized context distributions or reward augmentation terms significantly reduces skill diversity and precision. The staged MoME approach is also computationally efficient during inference, as each input is routed to an appropriate expert, parallelizing or serializing the computation as dictated by the gating structure (Celik et al., 2021, Mercurius et al., 13 Feb 2024).

5. Applications: Robotics, Prediction, and Beyond

MoME systems have been applied to a wide spectrum of movement-centric tasks, including:

• Robotic skill libraries: Robotic arms, simulated agents, and multi-limb robots use MoME frameworks to master a suite of non-trivial movement primitives, each tailored for a specific subtask or environmental configuration (Celik et al., 2021).
• Trajectory prediction: Modular MoME architectures cluster trajectory data, assign experts to subdomains such as rare maneuvers or safety-critical situations, and utilize a router for inference-time expert selection, thus boosting accuracy on long-tail events (Mercurius et al., 13 Feb 2024).
• Human motion and gait analysis: Multi-stage mixtures facilitate feature extraction at various body/limb abstraction levels, improving multi-task learning for psychological or biometric attribute estimation (Cǎtrunǎ et al., 6 Oct 2025).
• Locomotion and navigation: Hierarchical mixtures, as instantiated in multitask robot locomotion or traversability estimation, resolve gradient conflict between tasks and allow on-the-fly skill composition, migration, and adaptive expert fusion (Huang et al., 11 Mar 2025, He et al., 16 Sep 2025).

This broad utility arises from MoME’s staged, decomposable framework, which is adaptable and modular across discrete and continuous movement domains.

6. Algorithmic and Mathematical Underpinnings

Key equations frequently appearing in MoME literature include:

• Objective decomposition with augmented rewards:

$$R_{\text{aug}} = R(\theta, c) + \alpha \log \pi(\theta|o, c) + (\beta-\alpha) \log \pi(c|o)$$

• Per-expert lower bound optimization, context specialization, and expectation-maximization style updates for variational terms.
• Gating weight updates via softmax functions, routing input representations to expert nets:

$$a_t = \sum_{i=1}^N \hat{g}_i f_i(h_t), \quad \hat{g}_i = \mathrm{softmax}(g(h_t))[i]$$

• Curriculum and KL divergence regularization:

$-\beta\,\mathrm{KL}\big(\pi(c)\|\ p(c)\big)$

• Iterative expert addition and routing updates, ensuring on-demand scalability.

The model is trained through alternating maximization over per-expert RL objectives and updating gating priors and context distributions. In extension, additional modules may incorporate performance-driven router networks, auxiliary multi-task heads, or hierarchical segmentation for high-dimensional prediction settings.

7. Implications, Limitations, and Outlook

MoME frameworks offer several advantages over monolithic or flat MoE approaches:

• Versatility: Achieves broad and precise context coverage by modular composition.
• Precision: Experts trained on local contexts avoid mode averaging and learn more refined policies.
• Scalability: Allows dynamic addition of new skills without re-training the entire system.
• Interpretability: Facilitates inspection and manipulation of expert behaviors and gating dynamics.

However, practical deployment requires careful tuning of the curriculum mechanisms, reward shaping, expert parameterization, and gating scheme. Computational burden may grow with the number of experts, although inference costs are mitigated by context-sensitive routing. Theoretical challenges remain in optimal context partitioning, guarantees of expert orthogonality, and long-term stability in online adaptation scenarios.

In summary, Multi-Stage Mixture of Movement Experts architectures have demonstrated substantial improvements in robustness, accuracy, and behavioral diversity for challenging movement-based tasks. Ongoing research explores their integration with transfer learning, model-based elements, and real-world interactive systems for further gains in generalization and practical utility (Celik et al., 2021, Mercurius et al., 13 Feb 2024, Huang et al., 11 Mar 2025, Cǎtrunǎ et al., 6 Oct 2025).