OPINE-World: Programmatic World Modeling with Ontology-error-Prioritized Interactive Exploration

Abstract: Learning how an environment behaves from interaction is central to building agents that adapt to unfamiliar tasks. World models learned with deep networks are flexible but data-hungry and transfer poorly beyond their training distribution. Program-synthesized world models, written as source code by LLMs and refined through counterexample-guided inductive synthesis (CEGIS), are instead data-efficient and reusable, yet they have been demonstrated mainly on structured-state worlds with a given object vocabulary, and a single program search does not scale to pixel-rendered environments whose object structure must be hypothesized flexibly. We introduce OPINE-World, an LLM agent that learns an object-centric programmatic world model online from interaction. OPINE-World couples two cooperating agents in a loop of hypothesis and test, one acting in the environment and one synthesizing the model in code with replay verification and model-based planning, and it steers exploration with a Bayesian measure of object-type adequacy we call ontology error. We evaluate OPINE-World on ARC-AGI-3, a benchmark for skill-acquisition efficiency in which the object vocabulary, the goal, and the action semantics are withheld. OPINE-World solves 20 of 25 games without per-game training and reaches an action-efficiency score of 78.4 against the human baseline.

• The paper demonstrates that counterexample-guided LLM program synthesis and object-centric abstraction yield reusable, data-efficient world models.
• It details a dual-agent system that decouples acting and modeling, employing Bayesian ontology error to guide targeted exploration.
• Experimental results on ARC-AGI-3 show superior action efficiency and generalization compared to model-free and traditional neural approaches.

OPINE-World: Programmatic World Modeling with Ontology-error-Prioritized Interactive Exploration

Introduction and Motivation

OPINE-World proposes a principled bridge between programmatic world modeling and LLM-based agent architectures within interactive environments characterized by open-ended mechanics, sparse reward, and minimal prior knowledge. The principal aim is to efficiently induce reusable, object-centric transition models from raw interaction by coupling two specialized LLM-driven agents in a counterexample-guided synthesis (CEGIS) loop, with exploration explicitly prioritized via a Bayesian “ontology error” diagnostic. This responds directly to the observed limitations of deep-neural world models—namely, poor sample efficiency and limited out-of-distribution generalization—and to the lack of reusability and introspection in end-to-end agentic approaches.

Problem Setting and Framework

The environment is formalized as an object-oriented MDP, instantiated on the ARC-AGI-3 benchmark. The agent observes only raw $64 \times 64$ frames, and synthesizes a perception system to extract object records without access to ground-truth object identities. Crucially, the partition of objects into “mechanical types,” the goal, and the action semantics are all unknown and must be inferred online. The agent is evaluated for its action efficiency—minimizing actions taken to fully solve a sequence of levels—under a single-episode constraint per game.

OPINE-World Architecture

The Hypothesis-and-Test Loop

OPINE-World runs a continual loop Figure 1 in which:

Figure 1: The OPINE-World loop, coupling goal-directed acting, model synthesis and verification, planning, counterexample collection, ontology error diagnostics, and critic-driven model auditing.

1. The goal-directed agent acts in the live environment, records transitions, and probes hypotheses about unseen mechanics.
2. The world-model agent reads the transition buffer, synthesizes/updates an object-centric program (Python code) that must exactly replay all observed transitions.
3. A plan is computed over admitted (verified) models and executed step-wise, monitoring for mismatches (counterexamples).
4. The “ontology error” diagnostics guide targeted exploration to inform model refinement.

Object-Centric Factorization

The world model is a type-factored program, where each hypothesized object type is governed by a transition rule:

Figure 2: Decomposition of a grid state as object records, each associated with a key, hypothesized type, and attribute vector; type-factored rules generalize across isomorphic entities.

This allows rapid parameter sharing and knowledge transfer once type partitions are accurate. Importantly, the partition is itself inferred and is subject to local revision, supporting flexible correction and adaptation as object roles are clarified through ongoing experience.

Coordination via Specialized Agents

The decoupling of acting and modeling, realized as distinct LLM agents operating over a monotonic buffer, enables role specialization. The acting agent can pursue goals and probe model uncertainty, while the synthesizer focuses on code generation and counterexample-driven correction. Planner construction is bootstrapped only after initial level success and the transition model passes exact replay tests.

Verification by Exact State Replay

Unlike approaches relying on likelihood or heuristic error signals, OPINE-World employs a binary decision interface: a programmatic model is only admitted if it exactly reproduces every historical transition attribute-wise and object-wise. Counterexamples trigger model revision; synthesis is event-driven rather than periodic, economizing LLM calls and focusing computation on genuine errors.

Bayesian Ontology Error and Directed Exploration

The ontology error diagnostic is computed per-object as a noisy-OR of:

• Typing uncertainty (posterior entropy over possible roles/types).
• Effect uncertainty (entropy of the Dirichlet-smoothed effect distribution for a given type, action, and context row).

Object-level effect signatures are summarized Figure 3, supporting context refinement diagnostics Figure 4: as the context is refined (e.g., by splitting on observable features), rows that become deterministic indicate successful explanation, while residual mixture signals missing factors.

Figure 3: Example of effect signatures summarizing which attributes changed for each object transition.

Figure 4: Row concentration improves as before-state context features (e.g., object pixel state) are added, revealing deterministic substructure; persistent mixture indicates missing causal factors.

Aggregate ontology error Figure 5 supplies a cheap epistemic signal steering exploration toward unresolved object-mechanism relationships, guiding the acting agent toward informative interventions.

Figure 5: Aggregate ontology error $\eta_t$ falls as types and context features resolve; spikes indicate model repair, after which $\eta_t$ decreases toward zero.

Experimental Results on ARC-AGI-3

OPINE-World exhibits a decisive advantage in unsupervised, no-per-game-training generalization on ARC-AGI-3. Key outcomes:

• 20/25 games solved and 160/183 levels cleared without per-game training, surpassing a strong single-agent coding baseline (clears 14/25).
• 78.4 mean ARC-AGI-3 action-efficiency score vs. 63.8 for baseline1, often outperforming the human reference in raw actions to solve.
• Model-free and standard neural latent world models (MuZero/Dreamer families) solve zero games under the same constraints.

The action-efficiency margin is pronounced on “hard” games (6), with OPINE-World clearing them at ¼ the action cost of baseline1 and at ~2/3 the human cost, whereas baseline1 exhausts budgets with little progress. On games within baseline1’s reach, OPINE-World matches or slightly surpasses its efficiency.

Implications and Theoretical Contributions

From an algorithmic perspective, OPINE-World demonstrates that:

• Counterexample-guided LLM program synthesis, coupled with object-centric abstraction, yields data-efficient, reusable world models that outperform both black-box neural and monolithic agentic systems under minimal prior knowledge.
• Dynamic role assignment and explicit diagnostic signals (ontology error) enable adaptive exploration and localize model corrections, facilitating rapid online skill acquisition even with coarse initial abstractions.
• The exact-replay criterion provides hard guarantees of observational consistency, supplanting uncertain likelihood-based or heuristic scoring in non-differentiable, symbolic domains.

Practically, the architecture is modular: perception, transition modeling, and symbolic planning are strictly decomposed, clarifying the contribution of each and enabling robust diagnosis and system improvement.

Limitations and Future Directions

Notably, OPINE-World as currently instantiated assumes observable-Markov determinism; environments with persistent hidden state or high observation aliasing degrade the diagnostic signal, as irreducible ontology error persists and the replay criterion does not guarantee model completeness. The requisite object extraction is bootstrapped from raw image frames, but further robustness to complex perceptual ambiguity remains to be demonstrated. Planning scale is bounded by the synthesized forward searcher, leaving open improvements in efficient symbolic search and relational generalization.

Extensions to stochastic and partially observable environments are critical next steps, as is strengthening the model induction pipeline to accommodate richer forms of embodied perception.

Conclusion

OPINE-World establishes the viability and empirical strength of scalable, programmatic world modeling grounded in counterexample-guided synthesis and Bayesian-driven exploration. It not only significantly raises the bar for data-efficient, transferable skill acquisition under minimal prior structure but also delineates a precise, auditable interface between perception, abstraction, and planning. Future research should generalize this framework to less idealized observation regimes, exploit deeper forms of relational and causal inference, and test its limits in even more open-ended real-world domains.