Multi-scale Mixture of World Models for Embodied Agents in Evolving Environments

Abstract: Embodied agents operating in the real world require multi-scale reasoning and knowledge adaptation as conditions change. We identify two challenges in applying Mixture of Experts (MoE) to this setting: routing lacks an explicit notion of scale, preventing targeted updates at specific scales, and a uniform update policy cannot accommodate the different rates at which knowledge at each scale becomes outdated. We present MuSix, a framework that addresses both challenges through scale-aware world model mixture and evolution. A two-stage routing mechanism grounds scale selection in experiential distance, a measure of situational novelty inspired by Construal Level Theory: a meta-router first maps this quantity to a weight over continuous scale space, then per-scale base routers select world models within the identified scale. For adaptation, scale-dependent forgetting rates allow low-scale knowledge to refresh rapidly while high-scale abstractions persist, and gated inter-scale transfer maintains coherence across the hierarchy. Experiments on EmbodiedBench and HAZARD show that MuSix improves over state-of-the-art baselines on multi-scale reasoning and dynamic adaptation.

• The paper introduces a scale-aware framework, Oursol, that decomposes agent knowledge into hierarchical models for dynamic adaptation.
• It employs a two-stage routing system using experiential distance to select scale-specific world models based on varying levels of abstraction.
• Experimental results show significant performance gains in navigation and manipulation tasks, confirming robust online adaptation in evolving environments.

Multi-scale Mixture of World Models for Embodied Agents in Evolving Environments

Problem Formulation and Limitations of Standard MoE

This work addresses the challenge of developing embodied agents capable of effective multi-scale reasoning and online adaptation in non-stationary environments. Conventional Mixture of Experts (MoE) architectures facilitate modularization and targeted domain adaptation but fall short in two critical aspects for embodied agency: (1) routing is independent of knowledge scale, which disables selective adaptation of scale-specific knowledge, and (2) update mechanisms are homogeneous across experts, neglecting that lower-scale world knowledge (e.g., local physical dynamics) requires rapid refresh while abstract, higher-scale knowledge changes slowly.

Additionally, standard world model frameworks either remain monolithic or introduce hierarchy without explicit, measurable scale grounding. The work formalizes the need for scale-aware knowledge mixture and evolution, which underpins robust online reasoning for agents deployed in dynamic real-world settings.

Proposed Framework: Multi-scale Mixture-of-World Models (\textsc{Oursol})

The proposed solution, \textsc{Oursol}, introduces a multi-scale MoE framework, in which agent's knowledge is decomposed into a hierarchy of world model groups, each associated with distinct scales—concrete local dynamics to abstract reasoning. Central to the framework are two innovations:

Two-stage Scale-based Routing and Experiential Distance

A two-stage routing architecture decomposes routing into "scale determination" and "world model selection":

1. Meta-router: Computes weights over a continuous scale space, conditioning on an experiential distance signal—a psychologically and cognitively principled novelty measure derived from Construal Level Theory (CLT).
2. Per-scale Base Routers: Select specific world models within the targeted scale, enabling top-$k$ mixture selection.

Experiential distance is instantiated as a dynamically computed Mahalanobis distance in world model embedding space between the current observation and the agent's recent accumulation of experience, using exponentially decaying weights. This scalar aligns with CLT’s assertion that abstraction level should closely track the novelty or "distance" of current situations from prior experience.

The learning objective explicitly aligns the meta-router’s expected scale with normalized experiential distance, regularizes the concentration of routing across scale, and prevents degenerate solutions, ensuring the abstraction hierarchy is robust, interpretable, and adapts as intended.

Intra- and Inter-scale Knowledge Adaptation

World models are supplemented with scale-specific knowledge states. During online prediction, prediction errors or "knowledge increments" are:

• Dispatched to scale-appropriate world model groups based on meta-router output;
• Integrated with scale-dependent forgetting rates, so that low-scale (high volatility) world models rapidly refresh while high-scale (stable, slow-changing) abstractions persist;
• Propagated via gated cross-scale transfer, ensuring coherence and preventing information silos in the abstraction hierarchy. Gating parameters are learned during training, but only knowledge states are modified at test time, enabling continual adaptation.

Experimental Evaluation

Benchmarks and Metrics

The framework is validated on EmbodiedBench (multi-scale reasoning, manipulation, and navigation capabilities) and HAZARD (online adaptation under evolving environments such as fires and floods). The baselines include LLM-Planner, SayCanPay, FLARE, and a conventional MoE method lacking scale-awareness.

Numerical Outcomes

• On EmbodiedBench (Habitat), \textsc{Oursol} yields a 6.05 p.p. improvement over SayCanPay.
• On HAZARD (Fire), the method surpasses FLARE by 1.49 p.p. in the principal metric.
• In real-robot Franka Research 3 experiments, \textsc{Oursol} either outperforms or ties the strongest baselines and attains the highest worst-case success rates.
• Ablation confirms that both meta-router and intra-/inter-scale adaptation modules are indispensable; removing any significantly reduces downstream performance.

The framework generalizes across world model classes (e.g., RBFN, cognitive map), and the benefit holds robustly across architectural choices. Higher-dimensional scale spaces further improve performance via emergent specialization of axes for distinct novelty facets.

Theoretical and Practical Implications

By operationalizing a scale axis with cognitive grounding (CLT), the approach enables explicit, interpretable, and flexible mixture-of-world-models choices, unlike black-box latent expert selection. This mechanism supports test-time adaptation at appropriate knowledge scales with minimal interference—directly confronting catastrophic forgetting and over-stability issues pervasive in multi-scale and continual learning settings. It bridges cognitive science and ML modularity, opening avenues for cognitively inspired adaptation strategies for embodied intelligence. Practically, the method advances robustness in agents operating in unpredictable, evolving real-world domains.

Future Directions

Immediate extensions include automated discovery of world model group structure, deployment across broader robotic platforms, and scaling to even richer task domains. The approach also suggests potential synergies with larger and more capable VLM backbones, and motivates exploration of alternative abstract scale metrics beyond Mahalanobis-based experiential distance.

Conclusion

\textsc{Oursol} presents an explicit, scale-aware MoE architecture that tightly integrates multi-scale world model mixture and knowledge evolution for embodied agents, with strong empirical results on both reasoning and adaptation benchmarks. Its cognitively principled routing and adaptation mechanisms offer principled solutions to key limitations in conventional hierarchical and MoE frameworks, establishing a path toward more robust, modular, and adaptable embodied intelligence.