---
title: Search via Co-Evolving World Model
url: https://www.emergentmind.com/topics/search-via-co-evolving-world-model
type: topic
---

# Search via Co-Evolving World Model

Searching arXiv for recent papers on co-evolving world models and search-oriented agent frameworks.
“Search via Co-Evolving World Model” denotes a design pattern in which search or planning is mediated by a world model that is itself updated as the agent, search frontier, or environment distribution changes. In the cited literature, this pattern appears in several forms: one-step action lookahead with on-policy self-distillation in COMAP, intent-level search-tree editing in K-Search, entailment-graph planning in PathWise, neurosymbolic belief-state replanning in CoEx, virtual web-server simulation in WebEvolver, controlled parallel-world search evaluation in Mind-ParaWorld, joint latent-action/world-model training in CoLA-World, and environment-task co-evolution in Agent-World [2606.02372; 2602.19128; 2603.04751; 2504.21024; 2507.22281; 2601.20539; 2510.26433; 2604.18292]. Taken together, these works suggest a general search paradigm in which model-based anticipation, search-space restructuring, and data- or environment-driven adaptation are coupled rather than separated.

## 1. Core formalization

A recurring formal motif is to cast search as sequential decision making over an explicit or implicit world model. In COMAP, the world model \(M_\phi\) is an autoregressive LLM that models \(p_\phi(s_{t+1}\mid s_t,a)\), with outputs \(M(s_t,a)=\hat s_{t+1}\) and \(M_f(s_t,a)=\hat f_{t+1}\), while the policy \(\pi_\theta\) has a draft mode \(\pi_{\text{draft}}\) and a future-aware reflection mode \(\pi_{\text{ref}}\). Candidate actions \(a_t^i\sim \pi_{\text{draft}}(\cdot\mid s_t)\) are scored by a reliability term and a predicted feature or reward, with
\[
r(a_t^i) = - D_{KL}\!\left(q^{(T)}(\cdot\mid s_t,a_t^i)\,\|\,q^{(S)}(\cdot\mid s_t,a_t^i)\right)
\]
and
\[
a_t^* = \arg\max_i \left[r(a_t^i)+\gamma \hat f_{t+1}^i\right].
\]
This makes each decision step a one-step search over \(N\) candidates, where confidence and task utility are jointly optimized [2606.02372].

K-Search gives the phrase a particularly explicit formalization. It represents the search state \(S_t\) as a rooted, directed tree with closed nodes for visited states and open nodes for pending high-level actions, where each open node carries a parent program \(x_{\mathrm{parent}}\), a natural-language intent \(\delta\), and a priority estimate \(V(u)\in[0,1]\). The world model induces a distribution \(P_{\mathrm{model}}(S_{t+1}\mid S_t,a_t,x_t,o_t)\) that inserts new child hypotheses, updates priority scores, and prunes branches after concrete programs are instantiated and evaluated. The optimization target is
\[
J(x)=s\times \frac{p_{\mathrm{ref}}}{p(x)},
\]
with \(s\in\{0,1\}\) indicating correctness and \(p(x)\) the observed latency, under a finite budget \(B\) of kernel evaluations [2602.19128].

This body of work supports a common abstraction: the world model is not merely predictive. It is also a search operator that proposes, scores, filters, or restructures future possibilities. A plausible implication is that co-evolution is best understood not as a single algorithm, but as a coupling principle between search-state construction and model adaptation.

## 2. Search operators and planning topologies

The concrete search topology varies substantially across domains, even when the co-evolutionary principle is shared.

| System | Search state or world model | Search step |
|---|---|---|
| COMAP [2606.02372] | textual state \(s_t\), candidate actions, teacher–student reliability | one-step lookahead over \(N\) candidates |
| K-Search [2602.19128] | rooted directed tree with open and closed nodes | frontier action selection, local refinement, tree editing |
| PathWise [2601.20539] | entailment graph \(G_t=(V_t,E_t)\) | select \((S,\kappa)\), generate rollouts, insert \(v^+\) |
| CoEx [2507.22281] | adaptive belief state \(b_k=(m_k,l_k)\) | generate subgoal, execute, update belief, replan |
| Mind-ParaWorld [2603.04751] | Atomic Facts \(\mathcal F\) with PEM | only atomic queries yield deterministic evidence |

In PathWise, the search object is neither an action sequence nor a rollout tree in the conventional sense, but an entailment graph whose nodes store an executable heuristic program \(h\), a derivation rationale \(\kappa\), a concise algorithmic description \(d\), its performance \(P(h;D)\), and parent metadata \(PM\). The policy agent samples actions \(a_t=(S,\kappa)\), the world model agent generates \(N_w\) heuristic rollouts, and critic agents provide routed reflections that condition later decisions. Importantly, critic feedback does not update model weights; only the graph state \(G_t\) changes permanently [2601.20539].

CoEx instead elevates search to the subgoal level. A Planner \(P\) generates a natural-language subgoal \(e_k\), an Actor \(A\) executes it, and an Adaptive Belief State \(b_k=(m_k,l_k)\) is updated through deterministic symbolic parsing and LLM-based verification/synthesis. In this architecture, search proceeds by alternating subgoal generation and belief revision, so replanning is driven by changes in the world model rather than by explicit branch expansion [2507.22281].

Mind-ParaWorld is structurally different again: it is an evaluation environment for search agents in a controlled “parallel world.” Its ParaWorld Engine Model classifies queries as atomic or compound and only precise atomic queries that match an Atomic Fact in \(\mathcal F\) produce a truth entry; compound or low-quality queries return speculative or noisy passages with no fact leakage. This anti-shortcut mechanism converts search quality into a decomposition problem over query granularity [2603.04751].

A common misconception is that search via a co-evolving world model necessarily requires deep branching or expensive rollouts. COMAP explicitly describes its procedure as a “tiny one-step search,” and several other systems similarly concentrate capacity on state construction, search-space editing, or decomposition fidelity rather than depth alone [2606.02372].

## 3. Mechanisms of co-evolution

The defining property of the paradigm is that the world model changes as a consequence of the trajectories, rollouts, or environments produced by the current search process.

In COMAP, the executed transition \((s_t,a_t^*,s_{t+1}^{env})\) is appended to an on-policy buffer, and the student world model \(M_\phi^S\) is trained with a world-model self-distillation loss
\[
L_{WM}=\mathbb E_{(s,a,s')\in T}\!\left[-\log p_\phi^S(s'|s,a)+\eta\cdot \frac{1}{|s_{\text{pred}}|}\sum_j KL[\mathrm{sg}(q_j^T)\|q_j^S]\right].
\]
The policy is updated by maximum-likelihood imitation on the chosen action \(a_t^*\), plus regularization and auxiliary reflection losses in the full implementation. The intended effect is closed-loop refinement: as \(M\) improves on on-policy data, predictions become more reliable, which improves future action selection, which in turn yields better training data [2606.02372].

WebEvolver operationalizes co-evolution through alternating real interaction, policy fine-tuning, world-model next-observation training, and world-model–driven synthetic data generation. At iteration \(i\), successful real trajectories \(D_i\) are used both to fine-tune the agent \(M_{i-1}\to M_i\) and to train the world model \(W_{i-1}\to W_i\) on \((o_t,a_t)\to o_{t+1}\) prediction. The updated pair \((M_i,W_i)\) then synthesizes additional trajectories \(D_w\) in a virtual environment where the world model replaces the real web, and these trajectories augment subsequent policy training [2504.21024].

Agent-World moves the co-evolutionary locus outward, from transition prediction to environment synthesis. It maintains an evolving sequence \(\bigl(\mathcal E^{(r)},\mathcal X^{(r)},\pi_{\theta^{(r)}}\bigr)\), where \(\mathcal E^{(r)}\) is an environment pool, \(\mathcal X^{(r)}\) a task pool, and \(\pi_{\theta^{(r)}}\) the current policy. Each self-evolving round dynamically synthesizes fresh tasks, diagnoses failures with an LLM-based diagnosis agent \(\delta\), identifies weak environments \(W^{(r)}\), optionally complexifies the corresponding databases via \(\phi(\cdot)\), generates targeted tasks, and continues RL from \(\pi_{\theta^{(r)}}\) to obtain \(\pi_{\theta^{(r+1)}}\) [2604.18292].

CoLA-World shows that co-evolution can also occur inside the world-model stack itself. It first freezes the pre-trained world model and aligns a from-scratch inverse-dynamics model and VQ codebook during an 8 K-step warm-up, then jointly optimizes the latent-action model and world model for 52 K steps with a shared loss combining flow matching, VQ, and commitment terms. The paper presents the warm-up phase as critical for avoiding representational collapse and enabling successful joint learning [2510.26433].

These variants imply that “co-evolution” spans at least three levels: parameter-level adaptation of a predictive model, state-level growth of an external search structure, and curriculum-level evolution of environments and tasks. PathWise is a useful boundary case because its policy, world model, and critics co-evolve through the changing entailment graph and routed reflections, while model weights themselves remain fixed [2601.20539].

## 4. Domain instantiations and empirical behavior

The paradigm has been instantiated in embodied planning, Web navigation, tool use, automated heuristic design, GPU-kernel optimization, visual planning, search-agent evaluation, and scalable agent training.

COMAP reports gains across embodied task planning, Web navigation, and tool-use benchmarks, and states that it “consistently outperforms competitive baselines,” including “+16.75% relative improvement with Qwen3-4B.” Its analyses further report that the co-evolutionary loop improves world-model prediction accuracy over time and leads to more effective long-horizon decision-making [2606.02372].

K-Search evaluates on four complex kernels from FlashInfer—GQA Paged Decode, MLA Paged Prefill, MLA Paged Decode, and FP8 MoE—and reports final speedups over the FlashInfer baseline of 76.0, 57.4, 47.1, and 44.1, respectively, for an average of 56.13. The paper states that this is a “2.10× average improvement over OpenEvolve and 2.21× over ShinkaEvolve,” and that on the hardest MoE kernel it achieves a “14.3× higher speedup vs. OpenEvolve.” On the GPUMode TriMul leaderboard for H100, its Triton solution reaches “1 030 µs,” surpassing the prior best human-tuned CUDA entry at “1 074 µs” [2602.19128].

WebEvolver studies real-world web environments including Mind2Web-Live, WebVoyager, and GAIA-web. On WebVoyager, zero-shot Llama-3.3-70B is reported at “35.6%,” self-improvement adds “+6%” to approximately “42%,” and adding world-model–synthesized data yields an additional “+4%,” reaching approximately “46%.” In inference-time world-model look-ahead, depth 1 gives approximately “46.2%,” depth 2 approximately “51.4%,” and depth 3 approximately “49.5%,” while intrinsic world-model quality drops with rollout depth [2504.21024].

CoEx evaluates on ALFWorld, PDDL domains, and Jericho. Reported results include ALFWorld success rates of “ReAct 61.9, ExpeL 64.2, Reflexion 88.1, AdaPlanner 91.8, WALL-E 95.0, CoEx 93.3”; PDDL Gripper success/progress of “ReAct 65.0/89.5, HiAgent 75.0/89.9, CoEx 70.0/94.2”; Blocksworld “50.0/65.0, 50.0/68.3, CoEx 80.0/90.0”; and Jericho average progress “ReAct 36.4→CoEx 55.5” with success “10.0→25.0” [2507.22281].

PathWise reports that it “converges faster to better heuristics, generalizes across different LLM backbones, and scales to larger problem sizes” in combinatorial optimization. Its significance lies less in a single headline metric than in showing that co-evolving search memory can be encoded as a graph of derivations rather than as a flat population of candidates [2601.20539].

CoLA-World reports improvements in video prediction, real-action adaptation, and downstream visual planning. On RoboDesk visual planning with VP\(^2\), the two-stage pipeline achieves an average success rate of “6.9 %,” while CoLA-World reaches “13.1 %.” On LIBERO and RoboDesk real-action adaptation, it improves PSNR and reduces FVD relative to the two-stage baseline [2510.26433].

Mind-ParaWorld introduces MPW-Bench, “an interactive benchmark spanning 19 domains with 1,608 instances,” and shows that search agents are strong at evidence synthesis when complete information is available, but are limited by evidence collection and coverage in unfamiliar search environments, unreliable evidence sufficiency judgment, and when-to-stop decisions [2603.04751].

Agent-World evaluates across “23 challenging agent benchmarks.” On MCP-Mark, BFCL-V4, and \(\tau^2\)-Bench, Agent-World-8B reaches “8.9, 51.4, 61.8” and Agent-World-14B reaches “13.3, 55.8, 65.4,” while environment scaling from \(\{10,100,500,1000,2000\}\) environments yields average scores of “\{18.4%,29.5%,35.8%,38.1%,38.5%\}” across four representative domains [2604.18292].

## 5. Evaluation criteria, diagnostics, and recurrent bottlenecks

Because the world model is part of the search process rather than merely a simulator, evaluation typically separates process quality from end-task success. COMAP explicitly lists “World-model prediction accuracy (e.g. Delta-F1 on action-induced facts),” “Reliability calibration (how well \(r(a)\) correlates with actual outcome),” and “End-to-end task success rate” as metrics for search quality. Its reported trend is that “M’s Delta-F1 climbs steadily,” the “adoption rate of student predictions rises,” and “overall task success improves by double-digit percentages” [2606.02372].

Mind-ParaWorld provides a process-level diagnostic suite tailored to search agents. Final accuracy is measured with Pass@1, while process metrics include Fact Coverage Rate,
\[
\mathrm{FCR}=\frac{|\mathcal H|}{|\mathcal F|},
\]
Hit Rate,
\[
\mathrm{HitRate}=\frac{H}{T},
\]
and ToolCalls. The paper emphasizes that “insufficient coverage is almost always the primary failure mode,” and notes that agents “struggle to adapt queries over time” and to decide “when to stop searching” [2603.04751].

WebEvolver evaluates world-model fidelity intrinsically using “structural correctness (STR), similarity (Sim.), overall assessment (O/A) as scored by GPT-4o.” At rollout depth 1, reported values are “STR = 75.96%, Sim = 63.56%, O/A = 72.86%”; at depth 2, “O/A ~ 52.32%”; and beyond depth 3, “O/A < 45%.” The search implication is direct: deeper look-ahead increases compounding model error, so inference-time planning quality is bounded by rollout reliability [2504.21024].

Agent-World evaluates training dynamics under GRPO and reports that reward curves “ascend steadily” while actor entropy increases, which it interprets as continued exploration of new tool-use patterns rather than behavioral collapse. This is relevant because in a co-evolving regime, search degeneration can arise either from an inaccurate world model or from curriculum narrowing; entropy trends provide a partial diagnostic for the latter [2604.18292].

A recurring issue across the literature is reliability management. COMAP uses teacher–student KL divergence as an explicit confidence proxy; CoEx constrains textual belief updates to verified facts and allows later overwriting of erroneous inferences; WebEvolver tolerates hallucinations during training but relies on an evaluator during look-ahead; CoLA-World inserts a warm-up stage to prevent representational collapse before joint optimization [2606.02372; 2507.22281; 2504.21024; 2510.26433]. This suggests that co-evolution is beneficial only when paired with mechanisms that prevent self-reinforcement of model error.

## 6. Limitations, misconceptions, and open directions

Several limitations recur across implementations. In WebEvolver, deeper look-ahead does not monotonically improve performance: depth \(d=2\) gives the best trade-off, while for \(d\ge 3\) the world model becomes less reliable as errors compound. The same paper also reports only limited gains on “Level 2 (deep reasoning + web),” indicating that search quality alone does not resolve harder reasoning bottlenecks [2504.21024].

CoEx identifies a different failure regime: “abrupt or rapidly shifting environments may require faster belief resets or more aggressive pruning,” and “LLM inference may occasionally hallucinate in \(l_k\).” It also notes that scaling to very large tasks may demand memory–belief compression or stronger hierarchy beyond subgoals [2507.22281].

K-Search highlights computational constraints. Its current world-model updates rely on “vanilla in-context learning,” and the “volume of LLM calls may be a bottleneck”; the paper names “a distilled, lightweight surrogate model for \(P_{\mathrm{model}}\)” and extension to “multi-objective tuning (e.g. latency + energy)” as future directions [2602.19128].

Mind-ParaWorld frames an evaluative controversy rather than a deployment limitation. Static search benchmarks can suffer “difficulty collapse,” “fact drift,” and “confounded attribution,” making it hard to determine whether a model truly searched or simply exploited parametric memory. Its controlled parallel-world construction is therefore best understood as a response to a measurement problem in search-agent research rather than as a direct search algorithm [2603.04751].

Agent-World shows diminishing returns from environment scaling and from repeated self-evolution rounds: performance rises rapidly from 10 to 500 environments and then flattens, while “two rounds remain positive but smaller” than the first. This indicates that environment co-evolution is not unbounded and may require better diagnosis or diversification strategies at larger scales [2604.18292].

A final misconception is to equate co-evolution with parameter updating alone. PathWise demonstrates that critic feedback can condition future policy and world-model calls while “never” updating model weights, with only the graph state \(G_t\) changing permanently [2601.20539]. Conversely, CoLA-World shows the opposite extreme, where co-evolution occurs through fully joint optimization of a latent action model and a pre-trained world model after a carefully staged warm-up [2510.26433]. The literature therefore supports a broader definition: search via a co-evolving world model is any search regime in which the model of future possibilities is revised by the very process of searching, whether through weight updates, state-graph growth, belief revision, or environment synthesis.

Source: https://www.emergentmind.com/topics/search-via-co-evolving-world-model