Code2Worlds: Executable World Modeling

• Code2Worlds is a paradigm that models code as dynamic, executable world representations enabling simulation of transitions and state validation.
• It applies to diverse domains—ranging from Python trace debugging to GUI and physics-based simulations—ensuring verifiable and actionable outputs.
• The approach leverages rigorous metrics like trace fidelity and pass@1 scores to improve planning accuracy and debug complex state transitions.

Searching arXiv for the cited papers and related Code2World/Code World Model work. arXiv search query: Code2World OR "Code World Model" Code2Worlds denotes a family of formulations in which code is treated not merely as an output artifact but as a world representation, a transition model, or an executable substrate for simulation. The recent literature places this idea in several adjacent settings: code LLMs trained on execution traces and agentic trajectories, GUI world models that predict renderable code for the next screen, language-to-simulation systems that synthesize physically grounded 4D environments, and planners that translate rules and trajectories into simulators for reinforcement learning and games. Across these settings, the recurring objective is to replace purely implicit pattern matching with explicit computational state, executable dynamics, or renderable structure that can be verified, rolled out, or refined against an environment (team et al., 30 Sep 2025, Zheng et al., 10 Feb 2026, Wang et al., 14 May 2026, Zhang et al., 12 Feb 2026).

1. Conceptual foundations

In the code-reasoning line, a Code World Model is defined as a code LLM trained not only on static code but also on observation–action trajectories arising from a Python interpreter and from agentic interactions with repositories inside Docker containers. The model remains a next-token predictor, but the tokens encode world state and actions, so the learning problem approximates a transition function of the form $(o_t, a_t) \mapsto o_{t+1}$, where observations may include local variables, file contents, or test outputs, and actions may include executed lines, shell commands, and file edits (team et al., 30 Sep 2025).

A second line treats the simulator program itself as the world model. In that formulation, the world is not a latent transition $f_\theta(s_t, a_t) \mapsto s_{t+1}$ but executable code that declares bodies, joints, collision shapes, materials, terrains, robots, vehicles, fluids, controllers, and sensors, with the physics engine providing the numerical transition. The same idea appears in game-playing work, where the synthesized world model is Python code implementing state transition, legal move enumeration, observations, rewards, and termination checks, and in GUI work, where the world state is renderable HTML that deterministically maps to pixels through a renderer (Wang et al., 14 May 2026, Lehrach et al., 6 Oct 2025, Zheng et al., 10 Feb 2026).

This suggests that Code2Worlds is best understood as a shift from inert descriptions to executable or simulable state. The representation may be a Python trace, a DOM-like program, an OpenSpiel-compatible game simulator, or Blender/PyChrono code, but in each case the model is trained or prompted to predict how a world changes under actions rather than only how an artifact should look in isolation.

2. Code world models for program execution and agentic coding

The most explicit code-execution instantiation is CWM, a 32B dense decoder-only Transformer trained with a context size of up to 131k tokens. Its training recipe combines 8T tokens of general pre-training, 5T tokens of code world model mid-training, 100B tokens of supervised fine-tuning, and a multi-task RL phase using a modified GRPO variant over agentic software engineering, competitive programming, agentic coding, and math. The mid-training mixture includes Python execution traces, ForagerAgent Docker trajectories, GitHub PR and evolution trajectories, compiler IR, Triton kernels, and Lean data; the RL phase uses verifiable rewards such as passing hidden tests or mathematical equivalence. On general coding and math tasks, the final model reaches pass@1 scores of 65.8% on SWE-bench Verified with test-time scaling, 68.6% on LiveCodeBench, 96.6% on Math-500, and 76.0% on AIME 2024 (team et al., 30 Sep 2025).

Its core trace formalism serializes runtime execution as interleaved event tokens and JSON state. Given source code and a trace prefix, the model predicts the next executed line and the next state, formally as

$$p_\theta(a_{t+1}, s_{t+1} \mid x, a_{\le t}, s_{\le t}).$$

The interpreter world is encoded with events such as call, line, return, and exception; observations are effectively local-variable dictionaries plus current source line, while actions are next-line execution or state-transition labels. CWM can therefore be used as a neural Python interpreter or debugger, and the paper reports very high trace fidelity: valid trace format at least 99.6%, exact state-sequence match around 96–97%, exact action-sequence match around 96–98%, JSON validity 100%, key match at least 99.0%, and key+value match at least 97.9%. On CruxEval-Output, CWM SFT via full trace prediction reaches 87.3% pass@1, and the final CWM with language CoT reaches 94.3% (team et al., 30 Sep 2025).

The debugging literature shows where this paradigm fails. Dense runtime-state reveals are token-intensive, causing token-budget exhaustion on long execution histories, and failures disproportionately concentrate in string-valued state. On real-code benchmarks, strings account for 46.4% of outputs but 72.8% of non-truncation failures on CruxEval-O, and 17.2% of outputs but 44% of failures on HumanEval. In controlled composition tasks, non-string categories remain at 100% accuracy at depth 5, whereas string compositions degrade from 75% at depth 2 to 25% at depth 5. A separate long-horizon permutation benchmark shows that much apparent long-horizon degradation comes from incorrect action generation: when actions are teacher-forced, a Transformer-based CWM maintains approximately 90% accuracy at 128 steps (Rahmani, 7 Feb 2026).

A common misconception is that execution-trace supervision simply replaces chain-of-thought. The evidence is narrower and more specific: explicit traces provide typed, inspectable state transitions, but they do not eliminate representation bottlenecks, especially under BPE tokenization for strings, nor do they remove the need for planning over long horizons. The CWM results instead indicate that execution traces, agentic trajectories, and verifiable RL jointly create a stronger substrate for reasoning and tool use than static code pretraining alone (team et al., 30 Sep 2025, Rahmani, 7 Feb 2026).

3. Renderable code as a world representation for GUIs

In GUI modeling, Code2World replaces either textual state descriptions or direct image synthesis with renderable code generation. The central formulation predicts the next GUI state as code,

$$C_{t+1} = M_\theta(I_t, a_t, G), \qquad \hat I_{t+1} = R(C_{t+1}),$$

where a vision-language coder generates HTML-like code and a deterministic renderer converts it to the next screenshot. This design is meant to combine structural controllability with visual fidelity: code explicitly captures DOM hierarchy, geometry, and text, while rendering reconstitutes the visual state. To support this, the work constructs AndroidCode by translating Android GUI trajectories into HTML and refining synthesized code through a visual-feedback revision mechanism, yielding over 80K high-quality screen-action pairs. Training proceeds via SFT on code tokens and Render-Aware Reinforcement Learning with two rewards: a visual semantic reward and an action consistency reward, both computed after rendering with a VLM-as-a-Judge (Zheng et al., 10 Feb 2026).

Empirically, Code2World-8B achieves top-performing next-UI prediction and rivals GPT-5 and Gemini-3-Pro-Image. On AndroidControl, it reports $S_{\text{ad}}=94.28$, $S_{\text{id}}=88.64$, $S_{\text{ele}}=71.35$, and $S_{\text{lay}}=70.32$; on GUI Odyssey it maintains $S_{\text{ad}}=92.73$ and $S_{\text{id}}=78.22$. As a downstream simulator in navigation, it boosts Gemini-2.5-Flash by +9.5% on AndroidWorld, improving success rate from 41.4% to 50.9%. The ablation results are especially diagnostic: SFT alone establishes a usable simulator, $f_\theta(s_t, a_t) \mapsto s_{t+1}$0 alone improves visual quality, $f_\theta(s_t, a_t) \mapsto s_{t+1}$1 alone improves transition logic, and the full combination gives the strongest overall world-model behavior (Zheng et al., 10 Feb 2026).

A closely related system, gWorld, frames the same paradigm as visual world modeling via renderable code generation. It trains Qwen3-VL backbones to predict HTML/CSS-like code for the next GUI state, optionally preceded by a look-ahead reasoning trace, and renders the result in a browser. Across four in-distribution and two out-of-distribution benchmarks, gWorld 8B and 32B achieve average instruction accuracy of 74.9% and 79.6%, respectively, outperforming eight frontier open-weight models over 50.25x larger. Render failure is also much lower than for generic VLM baselines, and stronger world modeling correlates directly with better downstream policy performance (Koh et al., 2 Feb 2026).

This line makes the most explicit case that code can serve as a middle layer between semantics and perception. It is not merely a serialization convenience: the renderer supplies a deterministic observation model, the code makes layout and text explicit, and the agent can plan over simulated future screens without giving up visual grounding.

4. Executable simulation code for physical and virtual worlds

In physically grounded world generation, the code-to-world thesis becomes literal. One framework constructs physics-based world models through executable PyChrono scripts coordinated by a planning agent, a code agent, a visual review agent, and a physics analysis agent. The planning agent converts a natural-language prompt, optionally with an image, into a structured simulation plan; the code agent turns this plan into PyChrono code; execution produces trajectories, logs, and rendered frames; the visual review and simulation judge analyze semantic match and physical consistency; and the code is iteratively patched rather than regenerated from scratch. The world model is therefore the program plus the engine, with explicit simulator state and numerical integration rather than a latent neural dynamics model (Wang et al., 14 May 2026).

Code2Worlds for 4D generation extends the same idea to Blender and Infinigen, but adds a dual-stream architecture and a physics-aware closed loop. The Object Stream performs retrieval-augmented parametric generation for the focal dynamic object, using a procedural parameter library and a reference code library, then iteratively refines the object via a VLM-Critic. The Scene Stream separately handles atmospheric context, terrain morphology, ecosystem richness, and scene realization through a Planner, Resolver, and Realizer. A PostProcess Agent then scripts dynamics, and a VLM-Motion Critic evaluates rendered rollouts and feeds corrections back into the physics parameterization. On the Code4D benchmark, this framework reports a 41% SGS gain and 49% higher Richness than baselines, with a Failure Rate of 10%; it also achieves the best Motion Smoothness and Temporal Flickering among compared video systems (Zhang et al., 12 Feb 2026).

A much earlier precursor demonstrated the same pattern in VR and game engines. “Prompt-based creation of virtual worlds” integrated speech input, GPT-3 for semantic outcomes, Codex for C# script generation, Roslyn runtime compilation in Unity, Sketchfab asset retrieval, and Photon synchronization to allow users to “speak” world changes into existence during VR sessions. Its “Codex VR Pong” example turned collision semantics such as salmon plus knife into sushi, and used prompt-driven object replacement and runtime code execution as part of gameplay rather than only as a development tool (Roberts et al., 2022).

A recurring distinction in this literature is that executable simulators enforce constraints through the engine rather than through latent regularization. The physics engine handles non-penetration, joint constraints, gravity, and fluid–solid interaction; the code agent and critics tune parameters and repair structural mistakes. This suggests why these systems do better in scenarios where semantic correctness and physical consistency must coexist.

5. Planning, search, and decision making over synthesized worlds

Another branch of Code2Worlds uses code generation not primarily for rendering or interpretation, but for planning. In offline RL, a Code World Model is a Python Environment class with __init__, set_state, and step, synthesized from textual environment descriptions and offline trajectories. GIF-MCTS searches over candidate programs using three LLM actions—Generate, Improve, and Fix—inside a Monte Carlo Tree Search over multi-line code blocks. Candidate programs are scored by exact transition accuracy over held-out trajectories, and the final code model is then used by classical planners such as MCTS or CEM without further LLM calls. On APPS Competition, GIF-MCTS with Llama 3 70B reaches 28.3 ± 1.4% strict accuracy, exceeding WorldCoder and zero-shot CoT. On the Code World Models Benchmark, with Llama 3 and budget 50, it reports discrete-environment accuracy of 0.84 ± 0.03 and normalized return of 0.76 ± 0.03, versus 0.79 ± 0.04 and 0.60 ± 0.04 for WorldCoder; with GPT-4 Turbo and budget 10, the corresponding figures are 0.91 ± 0.08 and 0.81 ± 0.06 (Dainese et al., 2024).

The same pattern is extended to general game playing. There, the LLM translates natural-language rules and example trajectories into Python functions for apply_action, get_current_player, get_legal_actions, get_observations, and get_rewards, optionally plus heuristic value functions and hidden-state inference functions for imperfect-information games. The resulting simulator is used with MCTS or ISMCTS. Across 10 games, including 4 novel ones and both perfect- and imperfect-information settings, the synthesized-code agent outperforms or matches Gemini 2.5 Pro in 9 out of 10 games. In this formulation, verifiability arises from executable legal-move enumeration and termination checks, while strategic depth comes from classical search rather than direct next-move prompting (Lehrach et al., 6 Oct 2025).

These planning-oriented systems clarify an important property of Code2Worlds: once world knowledge has been compiled into executable code, reasoning can often be delegated to algorithmic search. That division of labor is explicit in both RL and general-game work: the LLM performs data-to-code translation, while planning and hidden-state reasoning are carried out on the synthesized simulator.

6. Evaluation regimes, limitations, and open problems

Evaluation in this area is necessarily heterogeneous because the represented worlds differ. CWM-style execution models are measured by trace fidelity, CruxEval-style output prediction, SWE-bench Verified, Terminal-Bench, HaltEval-prelim, and long-context benchmarks; GUI systems use action-adherence, action-identifiability, element alignment, layout integrity, and downstream navigation success; 4D physical systems use CLIP-derived alignment, GPT-4o-based SGS, Richness and HRS, VBench temporal metrics, and manually inspected Failure Rate; code-synthesized planners are judged by transition accuracy, unit-test pass rate, normalized return, and arena win rates (team et al., 30 Sep 2025, Zheng et al., 10 Feb 2026, Zhang et al., 12 Feb 2026, Dainese et al., 2024).

The limitations are likewise domain-specific but structurally related. In code-execution models, dense trace supervision causes token-budget exhaustion, and string-valued runtime state is brittle under subword tokenization; CWM itself notes that Python is the only language with explicit tracing and that single-step trace prediction still underperforms full reasoning on some output tasks (Rahmani, 7 Feb 2026, team et al., 30 Sep 2025). GUI world models incur rendering and VLM-judging costs, focus on Android mobile interfaces, and may hallucinate or misplace code-level elements despite strong next-state metrics (Zheng et al., 10 Feb 2026). Executable physics systems depend on simulator coverage, asset libraries, and prompt-engineered skill libraries; repair can be non-monotonic, runs may consume millions of tokens and tens of minutes, and unsupported sensors or rare physical regimes remain difficult (Wang et al., 14 May 2026, Zhang et al., 12 Feb 2026). Planning-by-code systems often assume deterministic, fully observable dynamics, or require carefully constructed trajectory-derived tests; performance is only as good as the correctness of the synthesized simulator (Dainese et al., 2024, Lehrach et al., 6 Oct 2025).

A common misconception is that Code2Worlds is a single architecture. The literature instead suggests a design space with at least three distinct substrates: explicit runtime state serialized as traces, renderable code compiled into visual observations, and full executable simulators backed by physics or game engines. A plausible implication is that future progress will come less from choosing one substrate universally than from matching the substrate to the world: typed trace states for program semantics, DOM-like code for GUIs, and engine-level programs for physical or strategic environments.

The future directions listed across these papers are unusually convergent. They include better integration of tracing with reasoning, neural debuggers that can jump or abstract over execution, richer environment models beyond CLI and text, support for other programming languages, combining executable world models with neural video models, parallel simulation and beam search over code variants, stronger hidden-state inference for partially observed domains, and more efficient state representations that retain verifiability without exhausting context windows (team et al., 30 Sep 2025, Rahmani, 7 Feb 2026, Wang et al., 14 May 2026, Zhang et al., 12 Feb 2026).