Code World Models (CWMs)

• Code World Models (CWMs) are techniques that leverage executable code to model environment dynamics, providing verifiable and interpretable simulations.
• They integrate program synthesis, iterative refinement, and hybrid neuro-symbolic methods to enhance planning accuracy and reduce computational overhead.
• CWMs have demonstrated high sample efficiency and adaptability across domains such as reinforcement learning, game playing, and robotics.

Code World Models (CWMs) refer to a family of techniques in machine learning and artificial intelligence in which executable code—typically in a high-level language like Python—serves as the formal, interpretable, and verifiable internal representation of an agent’s knowledge about environment dynamics, sensory inputs, transitions, and (optionally) reward structures. CWMs depart from traditional neural world models by making the model’s internal “simulation engine” explicit and accessible, enabling reasoning, planning, counterfactual analysis, and compositionality through mechanisms intrinsic to symbolic code execution. The CWM paradigm encompasses both neuro-symbolic models that synthesize code via LLMs and hybrid approaches that extract structured, causal, or object-centric models suitable for planning and policy induction.

1. Defining Characteristics and Rationale

CWMs are distinguished by the explicit use of code as a compact, executable world model. In a prototypical CWM, observations from the environment—together with a set of rules, natural language descriptions, or trajectories—are translated by an LLM or program synthesis system into code implementing key functions: state transition, legal action enumeration, reward evaluation, and termination predicates. This code, once validated by unit tests or offline trajectory replay, enables downstream agents and planners to simulate hypothetical transitions, run Monte Carlo rollouts, or perform value estimation with reliability and auditability (Tang et al., 19 Feb 2024, Dainese et al., 24 May 2024, Lehrach et al., 6 Oct 2025).

Key properties and motivations include:

• Verifiability: The code-as-model is inspectable, testable, and debuggable, allowing the explicit correction of errors and transparent model refinement.
• Sample and Compute Efficiency: By separating model learning (via code synthesis and trajectory-based unit testing) from planning (via code execution), CWMs avoid the repeated, costly calls to LLMs typical of “reasoning-on-the-fly” approaches (Tang et al., 19 Feb 2024).
• Generalization and Modularity: The interpretable code structure supports transfer learning, modular edits, and domain adaptation, as new or related environments can often be modeled by editing or composing existing code blocks (Tang et al., 19 Feb 2024, Lehrach et al., 6 Oct 2025).

2. Model Construction: Techniques and Algorithms

CWMs can be constructed through several related methodologies. Central themes include:

• Direct Program Synthesis via LLMs: Given a set of text rules and trajectory data, LLMs are prompted to synthesize Python code for stepwise environment simulation, legal move enumeration, and reward evaluation. Example: WorldCoder builds its model via code synthesis and iterative debugging, using trajectory-based test cases to ensure functional correctness; logical constraints ensure “optimism” (i.e., the code model allows for the discovery of reward trajectories) (Tang et al., 19 Feb 2024).
• Iterative Refinement, Improvement, and Bug-Fixing: Models such as GIF-MCTS utilize search-based strategies, organizing program refinement into a Monte Carlo Tree Search (MCTS) over code candidates. Actions include generating new code, improving based on unit-test failures (chain-of-thought debugging), and fixing execution errors, balancing exploration and exploitation via UCT-style heuristics (Dainese et al., 24 May 2024). The accuracy of synthesized models is defined as the average correctness on next-state, reward, and done signal over a suite of trajectories:

$$A = \frac{1}{N} \sum_{i=1}^N \left[\frac{1}{3} 1[s'_i = \hat{s}'_i] + \frac{1}{3} 1[r_i = \hat{r}_i] + \frac{1}{3} 1[d_i = \hat{d}_i]\right]$$

• Object- and Causal-Centric Extraction: Some frameworks (e.g., COMET) use object-centric perception modules to extract high-level state descriptors from observation or emulator memory, then employ symbolic regression to map these to underlying causal variables and update equations (Dillies et al., 9 Apr 2025). LLMs may then be employed to annotate variables with semantic meaning, enhancing interpretability.
• Auxiliary Model Inference: For partially observable or imperfect-information environments, auxiliary code functions for state inference (e.g., hidden action histories or state sampling) are synthesized alongside the core transition model. This supports planning approaches (ISMCTS, inference-based rollouts) in games and multi-agent environments (Lehrach et al., 6 Oct 2025).

3. Evaluation, Benchmarks, and Empirical Performance

Assessment of CWMs draws on both program synthesis metrics and downstream planning performance:

• Transition Accuracy: Synthesized code is tested for transition and reward accuracy across offline datasets or held-out trajectories. Accuracy is often the pass@1 metric (fraction of transitions where the model output matches ground-truth), as in the Code World Models Benchmark (CWMB) (Dainese et al., 24 May 2024).
• Planning and Task Success: In agentic tasks (reinforcement learning, general game playing), the CWM serves as the simulation engine for planning algorithms (e.g., MCTS, ISMCTS). Performance is quantified by normalized return (RL) or competitive win/loss rates (games). On CWMB, GIF-MCTS CWMs achieved average accuracy of 0.84 and normalized return of 0.76 in discrete environments, surpassing baselines such as WorldCoder (Dainese et al., 24 May 2024).
• Generalization to Novel Domains: CWMs are tested on out-of-distribution games or unseen tasks to evaluate adaptability. In game playing, CWMs matched or outperformed Gemini 2.5 Pro in 9 out of 10 games, including four newly synthesized games (Lehrach et al., 6 Oct 2025).
• Inference Speed: Once synthesized, CWMs execute at 4 to 7 orders of magnitude faster than direct LLM invocation for each planning step, permitting deep or extensive rollouts (Dainese et al., 24 May 2024).

4. Technical Foundations and Mathematical Formulation

CWMs formalize their learning objectives and planning pipelines through explicit constraints and loss functions:

• Consistency and Optimism Constraints: The WorldCoder agent requires synthesized code to be strictly consistent with collected (state, action, reward, next state, done) tuples (denoted as logical constraint $\phi_1$), and to admit a trajectory that achieves a goal state with positive reward (optimism, $\phi_2$) (Tang et al., 19 Feb 2024). Program synthesis is thus characterized by:

$$\forall (s, a, r, s', c, done): \ T(s, a) = s', \ R(s, a, c) = (r, done)$$

and the existence of a feasible reward trajectory for some $c$.

• Code Search and UCT Variants: In GIF-MCTS, program improvement employs a UCT variant for code candidate selection:

$$\text{UCT}(node_i) = v_i + C \sqrt{ \frac{\log N_{\text{parent}}}{n_{\text{same-action}} + \epsilon} }$$

where $v_i$ is node value (unit-test reward), $n_{\text{same-action}}$ counts previous expansions with the same action type, providing robust search under code synthesis (Dainese et al., 24 May 2024).

• Hybrid Neuro-symbolic Integration: In object-centric and causal-centric models, object extraction is performed via deep neural nets (e.g., Slot Attention, GNNs), and symbolic regression is applied to model variable transitions. For example:

$\text{Ball.y} = s_i - 14; \quad s_i = s_i + s_j$

encode causal relationships between object position and velocity (Dillies et al., 9 Apr 2025).

5. Applications and Domains

CWMs serve as the backbone for high-precision, interpretable planning across the following domains:

• Model-Based RL and Planning: By simulating hypothetical worlds, CWMs enable strategic, deep planning unattainable by implicit models. For RL environments, the agent can plan via code rollouts and optimize policy with reduced sample complexity (Dainese et al., 24 May 2024).
• General Game Playing: CWMs constructed for board and card games provide formal specifications for combinatorial planners, supporting legal move enforcement, inference in imperfect-information settings, and efficient heuristic evaluation (Lehrach et al., 6 Oct 2025).
• Agentic Software Engineering and Code Reasoning: Through interactive mid-training on interpreter and environment feedback, large LLMs with CWM capabilities can reason over program traces, support debugging, verification, and even simulate code execution in multi-turn agentic tasks (team et al., 30 Sep 2025).
• Multi-Modal, Vision-Language, and Robotic Domains: CWMs are used to structure multi-modal data (e.g., images with detailed attributes and bounding boxes) in a code-like format, improving alignment and reasoning across perception and action (Wang et al., 30 Sep 2024). Embodied agents may use mixture-of-world-models approaches, fusing latent and pixel-level features for improved control (Shang et al., 26 Sep 2025).
• Causal and Object-Centric RL: Algorithms such as COMET leverage CWMs to disentangle true causal dynamics from spurious correlations, enabling agents to plan and generalize more robustly (Dillies et al., 9 Apr 2025).

6. Limitations, Challenges, and Future Directions

CWMs face the following challenges:

• Synthesis Scalability and Correctness: Generating semantically and functionally correct code over complex domains remains nontrivial. Automated unit testing, iterative refinement strategies, and tree search mechanisms are employed to mitigate code errors, but the difficulty rises with environment complexity (Dainese et al., 24 May 2024, Lehrach et al., 6 Oct 2025).
• Expressiveness and Partial Observability: Extending CWMs to stochastic settings and partially observed environments requires auxiliary inference code and probabilistic modeling. Current CWMs often assume determinism and full observability; future work aims to synthesize probabilistic code and more powerful inference (Dainese et al., 24 May 2024, Lehrach et al., 6 Oct 2025).
• Integration with Multimodal and Structured Data: Hybrid models such as Graph World Models, which treat world-state as a graph with multi-modal node content, promise scalability and better representational alignment for code, vision, or other structured data (Feng et al., 14 Jul 2025).
• Safety and Trustworthiness: As with other world models, code-based approaches must avoid hallucinations, enforce physical and logical constraints, and provide robust uncertainty quantification, especially in safety-critical domains (Zeng et al., 12 Nov 2024).
• Causal Mediation and Control of Spurious Correlations: Structural causal modeling can be used to analyze and minimize the impact of spurious or direct (non-mediational) effects in code generation, guiding prompt and training design for more robust CWMs (Gupta et al., 7 Feb 2025).

7. Outlook and Research Ecosystem

With the release of open-weight, mid- and RL-trained CWM models (e.g., the 32B CWM LLM), the research community is now equipped with testbeds for agentic reasoning, neural debugging, and grounded code generation. Open-source pipelines, code repositories, and benchmarks (e.g., CWMB) facilitate rigorous evaluation and reproducibility (team et al., 30 Sep 2025, Dainese et al., 24 May 2024, Collu et al., 8 Jan 2024). As CWMs evolve, plausible implications include deeper integration with execution-based reasoning, more structured modular world model libraries, hybrid neuro-symbolic and probabilistic modeling for open-world and high-dimensional domains, and more widespread adoption in agentic planning, robotics, and scientific computing.

CWMs represent a convergence of program synthesis, classical simulation, causal modeling, and deep learning, forming a new foundation for verifiable, interpretable, and efficient artificial agents.