Foundation World Models in Predictive AI
- Foundation world models are learned predictive substrates that integrate representation learning, action-conditioned dynamics, and multimodal reasoning to support simulation, planning, and control.
- They follow canonical patterns such as latent-state simulation, explicit state factorization, and semantic grounding between embeddings and predictive states.
- Research applications span robotics, telecom, and climate science, highlighting both practical advancements and challenges in evaluation and reliability.
Foundation world models are learned predictive substrates that aim to support simulation, reasoning, planning, and adaptation across tasks, modalities, or domains. In the literature, the term spans at least three closely related programs: latent dynamics models descended from early world-model work in reinforcement learning (Ha et al., 2018); architectures that ground foundation-model semantics inside actionable predictive state spaces (Wang et al., 15 Jul 2025, Fu et al., 25 May 2026); and broader agendas that treat world models as persistent, compositional, and verifiable representations for embodied or cyber-physical intelligence (Gupta et al., 2024, Delgrange, 27 Feb 2026). This suggests that “foundation” does not denote a single architecture. Rather, it denotes a reusable predictive core that couples representation learning, action-conditioned dynamics, multimodal reasoning, and downstream control.
1. Conceptual scope and historical lineage
A standard historical reference point is the tripartite decomposition of visual model, memory/dynamics model, and controller introduced in early latent-model RL. In that formulation, a convolutional VAE compresses observations into a latent code, an MDN-RNN models stochastic latent transitions conditioned on actions, and a minimal controller maps latent state and memory to actions. The key demonstration was that an agent could be trained entirely inside a learned “dream” and then transferred back to the actual environment (Ha et al., 2018). Although this work was not foundation-scale, it established a durable recipe: learn a compact latent state, learn how it evolves under action, and use that learned simulator as a substrate for control.
Later literature broadened both the scale and the ambition of the concept. In robotics, “world foundation models” are described as large pre-trained generative video dynamics models whose broad physical priors are learned from massive video corpora, but which require specialization for precise manipulation (Huang et al., 3 Dec 2025). In multi-view robotic simulation, the term refers to DiT-based latent video models augmented for 3D consistency across cameras (Huang et al., 16 Jun 2026). In cyber-physical infrastructure, the same idea appears as a persistent, action-conditioned, uncertainty-aware model of telecom system dynamics that unifies prediction, planning, and orchestration (Zou et al., 8 Apr 2026). At the level of research agenda, foundation world models have also been defined as “persistent, structured representations of open environments” that support zero- or few-shot querying, specification-aware verification, and adaptation under novelty (Delgrange, 27 Feb 2026).
This broader usage shifts emphasis from mere next-observation prediction to a stronger bundle of properties: persistence across tasks, compositionality, semantic queryability, calibration, and usefulness for decision-making. A related perspective argues that embodied agents require not just broad multimodal models, but foundation veridical world models that can support interventions and counterfactuals across diverse systems (Gupta et al., 2024).
2. Canonical architectural patterns
Despite the diversity of terminology, several recurring architectural motifs can be identified. One is the latent-state simulator: encode observations into a compact state, propagate that state with an action-conditioned dynamics model, and attach a planner, controller, or actor-critic to the latent rollouts. This pattern is explicit in the original VAE–MDN-RNN–controller pipeline (Ha et al., 2018), in Dreamer-style latent control stacks such as FOUNDER’s RSSM-based world model (Wang et al., 15 Jul 2025), and in telecom formulations that treat the world model as a factored POMDP over controllable and exogenous dynamics (Zou et al., 8 Apr 2026).
A second motif is explicit factorization of state. Telecom World Models separate the hidden state into a Controllable System World and an Exogenous World, with asymmetric transition structure
so that operator actions directly affect only the controllable subsystem while the external world evolves stochastically (Zou et al., 8 Apr 2026). A related, though more control-oriented, factorization appears in TC-WM, where the compact latent is conceptually decomposed into a task-relevant physical block and a complementary semantic block, with alignment to proprioception and reconstruction back to the foundation-feature manifold (Fu et al., 25 May 2026).
A third motif is explicit grounding between foundation-model embeddings and world-model state. FOUNDER learns a mapping from InternVideo2 embeddings into the latent state space of a DreamerV3-style world model, so that text or video prompts become goal states inside the learned simulator rather than remaining external semantic descriptors (Wang et al., 15 Jul 2025). TC-WM makes a closely related argument from the opposite direction: frozen foundation-model embeddings are semantically rich but often poorly matched to downstream planning and control, so they should be treated as a semantic scaffold rather than as the final dynamical state (Fu et al., 25 May 2026).
3. Representations: pixels, foundation features, and structured worlds
Representation design is a central fault line in the field. One line of work keeps the predictive target close to raw observations. Another moves to semantically richer intermediate states. A third introduces explicit structure such as scene graphs, spatial cognitive maps, or geometric priors.
VFMF represents the world not as pixels but as DINOv2 feature states, arguing that pixel prediction is computationally intensive and not directly useful in many applications. It learns a VAE over multi-scale vision foundation model features and performs autoregressive latent flow matching, so that future states can be decoded into semantic segmentation, depth, surface normals, and even RGB (Boduljak et al., 12 Dec 2025). The core claim is that deterministic regression in feature space averages over plausible futures, whereas stochastic conditional generation in a compact generative latent can retain uncertainty while remaining more actionable than pixel-space prediction (Boduljak et al., 12 Dec 2025).
TC-WM addresses a complementary problem: foundation features may preserve excessive task-irrelevant detail. Its response is to linearly project frozen visual embeddings into a compact latent, align a subspace with the agent’s physical state via contrastive learning, and reconstruct the embedding space to preserve useful semantics (Fu et al., 25 May 2026). This architecture is explicitly motivated by reward-free offline settings, where latent geometry has to support planning without reward supervision or online interaction.
Other work introduces more explicit spatial structure. World2Mind builds structured spatial cognitive maps from 3D reconstruction and instance segmentation, synthesizes an Allocentric-Spatial Tree (AST) using elliptical parameters for top-down landmark layout, and reports that AST-structured text alone enables purely text-only foundation models to perform complex 3D spatial reasoning (Ruan et al., 10 Mar 2026). WorldAgents, by contrast, studies static 3D world synthesis from 2D image models using a director–generator–verifier loop and reconstructs accepted views into a 3D Gaussian Splatting scene (Erkoç et al., 20 Mar 2026). This suggests that some uses of “world model” in the literature concern static spatial consistency rather than action-conditioned dynamics.
In robotic video world models, multi-view consistency has become a representation problem in its own right. PAIWorld augments a DiT-based world foundation model with Geometry-Aware Cross-View Attention, Geometric Rotary Position Embedding, and Latent 3D-REPA, explicitly targeting cross-view object drift, depth inconsistency, and texture misalignment in manipulation settings (Huang et al., 16 Jun 2026).
4. Reasoning, planning, and control
The most practically consequential question is how world models are used. FOUNDER is a representative bridge architecture: a pretrained video-LLM supplies semantic embeddings, a learned grounding map converts those embeddings into world-model latent goal states, and behavior learning proceeds by imagined rollouts using a temporal-distance reward. On the DMC + Kitchen language-task benchmark, the full method reaches an average normalized score of $0.81$, compared with $0.60$ for GenRL, $0.57$ for WM-CLIP, $0.59$ for GenRL-TempD, and $0.52$ for FOUNDER without temporal distance (Wang et al., 15 Jul 2025). The paper’s central lesson is that foundation-model semantics become useful for control only after they are grounded into the state manifold where dynamics and imagination live.
AdaPower takes a different route. It starts from a pre-trained world foundation model, freezes the backbone, and inserts lightweight modules for test-time adaptation and long-horizon consistency, namely Temporal-Spatial Test-Time Training and Memory Persistence. The resulting specialist world model is embedded in an MPC loop that rolls out candidate action sequences proposed by a pre-trained VLA policy. On 10 unseen LIBERO tasks, the average success rate rises from $0.5$ for the pretrained VLA alone to $41.5$ with AdaPower (Huang et al., 3 Dec 2025). Here the world model is not merely a reward oracle or a synthetic-data generator; it is a decision-time predictive engine.
SENSEI shows another role for foundation models in world-model learning: semantic exploration. A pretrained VLM provides pairwise “interestingness” judgments over observations, those judgments are distilled into a reward model, and the world model learns to predict semantic reward from latent state. Exploration then maximizes both semantic reward and ensemble disagreement (Sancaktar et al., 3 Mar 2025). The contribution is indirect but important: foundation-model feedback changes what data the world model sees, and therefore what dynamics it learns.
A simpler but revealing setting is text-based GridWorld. There, the literature distinguishes Foundation World Models, used as zero-shot simulators for RL pretraining, from Foundation Agents, which act directly as policies. In the deterministic GridWorld, GPT-4 with achieved next-state and reward simulation accuracy, and FWM-based pretraining learned the optimal policy entirely in simulation. In the stochastic, partially observable setting, FWM pretraining remained useful even though stochastic simulation quality was imperfect (Sasso et al., 19 Sep 2025). The broader implication is that foundation world models and foundation agents are distinct design points, with the former becoming more attractive as partial observability and stochasticity increase.
5. Evaluation, failure modes, and contested claims
A recurring theme in the literature is that standard predictive metrics are often misaligned with the actual function of a world model. For safety prediction in autonomous robots, image-wide MSE and SSIM were argued to be inadequate because safety depends on small but semantically crucial state changes. The proposed alternative was a centroid-based state metric, together with F1 and false-positive rate on binary safety prediction. On Lunar Lander, a standard VAE + MDN-LSTM world model had false positive rate $0.81$0 across all horizons and input lengths, whereas SAM + GPT-3.5 achieved $0.81$1 and $0.81$2 at $0.81$3, $0.81$4 (Mao et al., 2024). The methodological point is broader than the benchmark itself: a world model should be evaluated in the state variables that matter for control, safety, or planning, not only in observation space.
Another major controversy concerns whether high predictive accuracy implies learning a genuine world model. The inductive-bias probe developed for orbital mechanics, lattice dynamics, and Othello is explicitly skeptical. It shows that models can obtain $0.81$5 on held-out orbital trajectory prediction and still fail to adapt according to Newtonian state structure on downstream tasks such as force prediction (Vafa et al., 9 Jul 2025). The authors’ interpretation is that these models behave as if they learn task-specific heuristics or coarsened state abstractions rather than the underlying world model. This directly challenges the common assumption that next-token or next-step success is sufficient evidence of mechanistic understanding.
Failure modes also arise inside learned simulators themselves. In the original dream-training setting, low sampling temperature made the simulated environment easy to exploit: at $0.81$6, the virtual score reached $0.81$7, but real-environment transfer fell to $0.81$8, below a random-policy baseline; at $0.81$9, virtual score dropped to $0.60$0 while real transfer improved to $0.60$1 (Ha et al., 2018). In action-centric image editing, copying the source image remained a recurrent failure mode, which motivated dynamics-model verification and recognition-weighted loss (Qiu et al., 6 Jun 2025). In multimodal reasoning benchmarks, current MFMs remain far from robust world-model behavior: on MMWorld, GPT-4o averages $0.60$2, while on VLM4D the best proprietary model, Gemini-2.5-Pro, reaches $0.60$3 against human performance of $0.60$4 (He, 4 Oct 2025).
These results support a cautious view. World modeling is not reducible to photorealistic generation, high next-token likelihood, or strong zero-shot VQA. The literature repeatedly returns to counterfactual reasoning, temporal understanding, controllability, calibrated uncertainty, and state-grounded evaluation as the harder tests.
6. Domain-specific instantiations and research agenda
The term “foundation world model” has also become a domain-specific design principle. In Earth and climate science, the ideal Earth foundation model is characterized by eleven features: geolocation embedding, balanced geographical representations, scale awareness, wavelength embedding, the time variable, multisensory input, task-agnostic adaptation, carbon minimization, uncertainty quantification, physical consistency, and AI assistants (Zhu et al., 2024). In this domain, a foundation world model is effectively an Earth-system representation spanning space, time, sensors, and tasks, rather than a narrow forecaster.
In embodied AI, a stronger normative claim has been advanced: useful foundation world models must be veridical, meaning they should support interventions and counterfactuals rather than only observational prediction (Gupta et al., 2024). A related open-world agenda proposes persistent structured representations that integrate learnable reward models from specifications, adaptive formal verification, online abstraction calibration, and verifier-guided test-time synthesis (Delgrange, 27 Feb 2026). These perspectives push the concept beyond model-based RL toward a substrate for learning, reasoning, adaptation, and certification.
Infrastructure and urban modeling exhibit parallel developments. Telecom World Models unify a Field World Model, a Control/Dynamics World Model, and a TelecomGPT orchestration layer; in a network-slicing proof of concept, the full three-layer pipeline reaches average $0.60$5 and $0.60$6, outperforming single-world baselines (Zou et al., 8 Apr 2026). BuildingWorld provides about five million LoD2 building models from 44 cities across five continents, together with real and simulated airborne LiDAR, meshes, and wireframes, explicitly to support urban-scale foundation modeling and analysis (Huang et al., 9 Nov 2025).
Taken together, these strands suggest a converging research agenda. Foundation world models are moving toward semantically grounded latent states, explicit treatment of action and uncertainty, stronger 3D and multi-view consistency, and tighter interfaces between predictive modeling, planning, and verification. A plausible implication is that the decisive advances will come less from a single universal architecture than from reusable predictive substrates that can be grounded, calibrated, specialized, and queried across domains.
