WorldCoder: Code-Based World Modeling

• WorldCoder is a model-based agent paradigm that uses an executable codebase to simulate world dynamics and reward structures.
• It employs a bandit-driven synthesis protocol with optimism constraints to refine symbolic programs for efficient exploration.
• Applications span gridworlds, 3D/4D simulations, and GUI environments, showing significant improvements over neural or prompt-driven models.

WorldCoder is a model-based agent paradigm in which an explicit, executable codebase serves as the world model, enabling code-level reasoning, planning, and knowledge transfer across interactive environments. Unlike end-to-end neural models or pure prompt-driven agents, WorldCoder synthesizes and maintains symbolic programs—typically in Python—that implement both the transition and reward structure of the agent's world, grounding all internal inference and exploration in interpretable, compositional code (Tang et al., 2024). This approach has generalized into a broader methodology for generating and simulating complex virtual worlds (static and dynamic), controlling both geometry and physics, from language or multimodal input, as seen in 3D, 4D, and GUI domains (Zhang et al., 12 Feb 2026, Liu et al., 22 Jul 2025, Zheng et al., 10 Feb 2026).

1. Symbolic Program World Models

At the core of the original WorldCoder architecture is a pair of code modules $(\hat{T}, \hat{R})$:

• $\hat{T}: S \times A \to S$: a transition function mapping states and actions to successor states.
• $$\hat{R}: C \to (S \times A \times S \to \mathbb{R} \times \{0,1\})$$: a (possibly contextual) reward function assigning rewards and terminal flags.

These programs jointly encode the agent's environment and reward logic. An LLM (e.g., GPT-4) is prompted to generate or refine this model so that it simultaneously (a) explains observed experience tuples and (b) remains optimistic about possible reward-achieving plans. Explicit formal requirements are:

• Fit-data constraint $\varphi_1(D, \hat{T}, \hat{R})$: the code must explain all $(s, a, r, s', c, d) \in D$.
• Optimism constraint $\varphi_2(s_0, c, \hat{T}, \hat{R})$: there must exist a plan $(a_1, s_1, \dots, a_\ell, s_\ell)$ with $r > 0$, i.e., the model must admit a successful policy (Tang et al., 2024).

The agent alternates between (i) simulating world dynamics and planning within $P = (\hat{T}, \hat{R})$ and (ii) interacting in the real environment, updating the replay buffer and triggering refinement only as needed.

2. Program Synthesis, Optimism, and Bandit-Guided Refinement

Model refinement in WorldCoder follows a bandit-driven synthesis protocol. Each candidate program (represented as an "arm") receives a Beta prior based on the fraction $h(P)$ of the data it explains. At each iteration, the agent samples from these posteriors to select which candidate to refine via the LLM. New programs are generated and scored as to whether they satisfy $\varphi_1$ and $\varphi_2$; priors are updated accordingly, and the process repeats until constraints are met.

The central mathematical feature is the explicit coupling of optimism under uncertainty (by requiring that $\varphi_2$ hold), which ensures exploration is directed towards finding counterexamples when the world model is incorrect, avoiding random search (Tang et al., 2024). This approach leads to efficient learning in sparse-reward or hard-exploration domains and amortizes LLM usage since only data-inconsistent or pessimistic models trigger expensive model edits.

3. Applications and Benchmark Performance

WorldCoder was evaluated on gridworlds including Sokoban and MiniGrid. The protocol is as follows:

1. Initialize $D \leftarrow \emptyset$ and $P = (\text{null}, \text{null})$.
2. For each episode, update $D$ with $(s,a,r,s',c,d)$, and only when $P$ fails $\varphi_1$ or $\varphi_2$ invoke LLM-driven refinement.
3. Use depth-limited value iteration for planning over $P$; use $\epsilon$-greedy exploration to ensure coverage.
4. Transfer across tasks/environments by retrieving and editing code $P$.

Empirical results include:

• Sokoban: correct model learned in $\sim 50$ steps, then zero-shot solving of new levels. By contrast, PPO requires $>10^6$ steps; ReAct agents require $O(T)$ LLM calls per episode.
• MiniGrid: curriculum transfer experiments show prior code dramatically reduces exploration in new, structurally similar tasks.
• Ablations confirm $\varphi_2$ (optimism) dramatically boosts sample efficiency and policy discovery in sparse-reward tasks (Tang et al., 2024).

4. Beyond Symbolic Coding: WorldCoder in 3D, 4D, and GUI Generation

The WorldCoder paradigm generalizes to "code as world model" systems in geometry and simulation:

• 4D World Generation: In "Code2Worlds," language-to-code generation produces simulation code $(C)$ that, when executed in a physics engine (e.g. Blender), creates 4D rollout $W_{4D} = \{O, E, t\}$, evaluated for semantic and physical alignment. The system employs a dual-stream architecture (object stream and environment stream), iterative VLM-based self-critique, and a physics-aware closed-loop (dynamic code refinement until VLM-Motion critic validates physical fidelity) (Zhang et al., 12 Feb 2026).
• Large-Scale 3D Earth Generation: "EarthCrafter" extends WorldCoder principles to photorealistic, geographically-plausible Earth synthesis at kilometer scale. A dual VAE/diffusion architecture separately codes geometry and texture, using massive multi-view, multi-terrain datasets with semantic priors. Applications include semantic-guided city layout and unconditional terrain synthesis (Liu et al., 22 Jul 2025).
• GUI World Modeling: "Code2World" treats GUI state prediction and simulation as renderable code generation. Via extensive paired screenshot-HTML datasets and render-aware RL, agents achieve high fidelity next-UI prediction and boost downstream navigation policies (e.g., 9.5% higher AndroidWorld success with plug-in world models) (Zheng et al., 10 Feb 2026).

5. Evaluation and Comparative Metrics

WorldCoder and WorldCoder-inspired systems are evaluated through metrics appropriate to their respective domains. In the agent/planning setting:

• Sample efficiency (steps to solve).
• Compute efficiency (LLM tokens or calls).
• Fraction of explained datapoints $h(P)$.
• Goal-reaching ablations: effect of $\varphi_2$ (optimism).

For generative world modeling:

• Code2Worlds: Semantic alignment (CLIP/GPT-4-based), Fine-grained Geometry Score (SGS), Scene Richness, Physics Failure Rate, Video Stability. SGS and Richness gains of 41% and 49% over prior methods, respectively, and >20% lower physics failure rates.
• Code2World: Next-UI prediction benchmarks (Action Adherence, Action Identifiability, Element Alignment, Layout Integrity), with in-domain $S_{ad}=94.28$ and out-of-domain $S_{ad}=92.73$, rivaling closed models like GPT-5 and Gemini-3-Pro-Image (Zheng et al., 10 Feb 2026).
• EarthCrafter: Large-scale structural and textural plausibility metrics, semantic-guided condition fidelity, and infinite extensibility across terrain scales (Liu et al., 22 Jul 2025).

6. Extensions, Domain Transfer, and Limitations

WorldCoder frameworks support substantial knowledge transfer. Existing code modules $(\hat{T}, \hat{R})$ or factory templates (for 3D/4D) are retrieved and minimally edited to support new environments or goals. This enables amortization of LLM inference cost and rapid adaptation in curriculum or transfer settings (Tang et al., 2024).

Known limitations include:

• Computational cost: iterative rendering and VLM critic loops (in 4D/3D settings) induce significant latency/budget requirements.
• Library coverage: results depend on the completeness of procedural parameter and code template repositories.
• LLM-induced artifacts: out-of-distribution prompts can provoke hallucinations or invalid code unless adequate data priors and critic integration are present.

By explicitly formalizing world dynamics and semantics as code, and integrating model editing, planning, simulation, and self-critique, WorldCoder defines a unifying framework for interpretable, transferable, and sample-efficient world modeling across a diverse set of domains (Tang et al., 2024, Zhang et al., 12 Feb 2026, Liu et al., 22 Jul 2025, Zheng et al., 10 Feb 2026).