Search via Co-Evolving World Model

Updated 5 July 2026

The paper shows that search via a co-evolving world model integrates predictive modeling and search operations, where the model updates in real time based on agent actions and environmental changes.
It employs diverse mechanisms such as one-step lookahead, search-tree editing, and belief state replanning across frameworks like COMAP, K-Search, and CoEx to refine decision-making.
Empirical evaluations reveal consistent improvements in task success and efficiency, reporting gains like +16.75% relative improvement and 2.1× speedup over traditional baselines.

Searching arXiv for papers on co-evolving world models and search-oriented agent frameworks. “Search via Co-Evolving World Model” denotes a design pattern in which search or planning is mediated by a world model that is itself updated as the agent, search frontier, or environment distribution changes. In the cited literature, this pattern appears in several forms: one-step action lookahead with on-policy self-distillation in COMAP, intent-level search-tree editing in K-Search, entailment-graph planning in PathWise, neurosymbolic belief-state replanning in CoEx, virtual web-server simulation in WebEvolver, controlled parallel-world search evaluation in Mind-ParaWorld, joint latent-action/world-model training in CoLA-World, and environment-task co-evolution in Agent-World (Liu et al., 1 Jun 2026, Cao et al., 22 Feb 2026, Chen et al., 5 Mar 2026, Fang et al., 23 Apr 2025, Kim et al., 29 Jul 2025, Gungordu et al., 28 Jan 2026, Wang et al., 30 Oct 2025, Dong et al., 20 Apr 2026). Taken together, these works suggest a general search paradigm in which model-based anticipation, search-space restructuring, and data- or environment-driven adaptation are coupled rather than separated.

1. Core formalization

A recurring formal motif is to cast search as sequential decision making over an explicit or implicit world model. In COMAP, the world model $M_\phi$ is an autoregressive LLM that models $p_\phi(s_{t+1}\mid s_t,a)$ , with outputs $M(s_t,a)=\hat s_{t+1}$ and $M_f(s_t,a)=\hat f_{t+1}$ , while the policy $\pi_\theta$ has a draft mode $\pi_{\text{draft}}$ and a future-aware reflection mode $\pi_{\text{ref}}$ . Candidate actions $a_t^i\sim \pi_{\text{draft}}(\cdot\mid s_t)$ are scored by a reliability term and a predicted feature or reward, with

$r(a_t^i) = - D_{KL}\!\left(q^{(T)}(\cdot\mid s_t,a_t^i)\,\|\,q^{(S)}(\cdot\mid s_t,a_t^i)\right)$

and

$a_t^* = \arg\max_i \left[r(a_t^i)+\gamma \hat f_{t+1}^i\right].$

This makes each decision step a one-step search over $p_\phi(s_{t+1}\mid s_t,a)$ 0 candidates, where confidence and task utility are jointly optimized (Liu et al., 1 Jun 2026).

K-Search gives the phrase a particularly explicit formalization. It represents the search state $p_\phi(s_{t+1}\mid s_t,a)$ 1 as a rooted, directed tree with closed nodes for visited states and open nodes for pending high-level actions, where each open node carries a parent program $p_\phi(s_{t+1}\mid s_t,a)$ 2, a natural-language intent $p_\phi(s_{t+1}\mid s_t,a)$ 3, and a priority estimate $p_\phi(s_{t+1}\mid s_t,a)$ 4. The world model induces a distribution $p_\phi(s_{t+1}\mid s_t,a)$ 5 that inserts new child hypotheses, updates priority scores, and prunes branches after concrete programs are instantiated and evaluated. The optimization target is

$p_\phi(s_{t+1}\mid s_t,a)$ 6

with $p_\phi(s_{t+1}\mid s_t,a)$ 7 indicating correctness and $p_\phi(s_{t+1}\mid s_t,a)$ 8 the observed latency, under a finite budget $p_\phi(s_{t+1}\mid s_t,a)$ 9 of kernel evaluations (Cao et al., 22 Feb 2026).

This body of work supports a common abstraction: the world model is not merely predictive. It is also a search operator that proposes, scores, filters, or restructures future possibilities. A plausible implication is that co-evolution is best understood not as a single algorithm, but as a coupling principle between search-state construction and model adaptation.

2. Search operators and planning topologies

The concrete search topology varies substantially across domains, even when the co-evolutionary principle is shared.

System	Search state or world model	Search step
COMAP (Liu et al., 1 Jun 2026)	textual state $M(s_t,a)=\hat s_{t+1}$ 0, candidate actions, teacher–student reliability	one-step lookahead over $M(s_t,a)=\hat s_{t+1}$ 1 candidates
K-Search (Cao et al., 22 Feb 2026)	rooted directed tree with open and closed nodes	frontier action selection, local refinement, tree editing
PathWise (Gungordu et al., 28 Jan 2026)	entailment graph $M(s_t,a)=\hat s_{t+1}$ 2	select $M(s_t,a)=\hat s_{t+1}$ 3, generate rollouts, insert $M(s_t,a)=\hat s_{t+1}$ 4
CoEx (Kim et al., 29 Jul 2025)	adaptive belief state $M(s_t,a)=\hat s_{t+1}$ 5	generate subgoal, execute, update belief, replan
Mind-ParaWorld (Chen et al., 5 Mar 2026)	Atomic Facts $M(s_t,a)=\hat s_{t+1}$ 6 with PEM	only atomic queries yield deterministic evidence

In PathWise, the search object is neither an action sequence nor a rollout tree in the conventional sense, but an entailment graph whose nodes store an executable heuristic program $M(s_t,a)=\hat s_{t+1}$ 7, a derivation rationale $M(s_t,a)=\hat s_{t+1}$ 8, a concise algorithmic description $M(s_t,a)=\hat s_{t+1}$ 9, its performance $M_f(s_t,a)=\hat f_{t+1}$ 0, and parent metadata $M_f(s_t,a)=\hat f_{t+1}$ 1. The policy agent samples actions $M_f(s_t,a)=\hat f_{t+1}$ 2, the world model agent generates $M_f(s_t,a)=\hat f_{t+1}$ 3 heuristic rollouts, and critic agents provide routed reflections that condition later decisions. Importantly, critic feedback does not update model weights; only the graph state $M_f(s_t,a)=\hat f_{t+1}$ 4 changes permanently (Gungordu et al., 28 Jan 2026).

CoEx instead elevates search to the subgoal level. A Planner $M_f(s_t,a)=\hat f_{t+1}$ 5 generates a natural-language subgoal $M_f(s_t,a)=\hat f_{t+1}$ 6, an Actor $M_f(s_t,a)=\hat f_{t+1}$ 7 executes it, and an Adaptive Belief State $M_f(s_t,a)=\hat f_{t+1}$ 8 is updated through deterministic symbolic parsing and LLM-based verification/synthesis. In this architecture, search proceeds by alternating subgoal generation and belief revision, so replanning is driven by changes in the world model rather than by explicit branch expansion (Kim et al., 29 Jul 2025).

Mind-ParaWorld is structurally different again: it is an evaluation environment for search agents in a controlled “parallel world.” Its ParaWorld Engine Model classifies queries as atomic or compound and only precise atomic queries that match an Atomic Fact in $M_f(s_t,a)=\hat f_{t+1}$ 9 produce a truth entry; compound or low-quality queries return speculative or noisy passages with no fact leakage. This anti-shortcut mechanism converts search quality into a decomposition problem over query granularity (Chen et al., 5 Mar 2026).

A common misconception is that search via a co-evolving world model necessarily requires deep branching or expensive rollouts. COMAP explicitly describes its procedure as a “tiny one-step search,” and several other systems similarly concentrate capacity on state construction, search-space editing, or decomposition fidelity rather than depth alone (Liu et al., 1 Jun 2026).

3. Mechanisms of co-evolution

The defining property of the paradigm is that the world model changes as a consequence of the trajectories, rollouts, or environments produced by the current search process.

In COMAP, the executed transition $\pi_\theta$ 0 is appended to an on-policy buffer, and the student world model $\pi_\theta$ 1 is trained with a world-model self-distillation loss

$\pi_\theta$ 2

The policy is updated by maximum-likelihood imitation on the chosen action $\pi_\theta$ 3, plus regularization and auxiliary reflection losses in the full implementation. The intended effect is closed-loop refinement: as $\pi_\theta$ 4 improves on on-policy data, predictions become more reliable, which improves future action selection, which in turn yields better training data (Liu et al., 1 Jun 2026).

WebEvolver operationalizes co-evolution through alternating real interaction, policy fine-tuning, world-model next-observation training, and world-model–driven synthetic data generation. At iteration $\pi_\theta$ 5, successful real trajectories $\pi_\theta$ 6 are used both to fine-tune the agent $\pi_\theta$ 7 and to train the world model $\pi_\theta$ 8 on $\pi_\theta$ 9 prediction. The updated pair $\pi_{\text{draft}}$ 0 then synthesizes additional trajectories $\pi_{\text{draft}}$ 1 in a virtual environment where the world model replaces the real web, and these trajectories augment subsequent policy training (Fang et al., 23 Apr 2025).

Agent-World moves the co-evolutionary locus outward, from transition prediction to environment synthesis. It maintains an evolving sequence $\pi_{\text{draft}}$ 2, where $\pi_{\text{draft}}$ 3 is an environment pool, $\pi_{\text{draft}}$ 4 a task pool, and $\pi_{\text{draft}}$ 5 the current policy. Each self-evolving round dynamically synthesizes fresh tasks, diagnoses failures with an LLM-based diagnosis agent $\pi_{\text{draft}}$ 6, identifies weak environments $\pi_{\text{draft}}$ 7, optionally complexifies the corresponding databases via $\pi_{\text{draft}}$ 8, generates targeted tasks, and continues RL from $\pi_{\text{draft}}$ 9 to obtain $\pi_{\text{ref}}$ 0 (Dong et al., 20 Apr 2026).

CoLA-World shows that co-evolution can also occur inside the world-model stack itself. It first freezes the pre-trained world model and aligns a from-scratch inverse-dynamics model and VQ codebook during an 8 K-step warm-up, then jointly optimizes the latent-action model and world model for 52 K steps with a shared loss combining flow matching, VQ, and commitment terms. The paper presents the warm-up phase as critical for avoiding representational collapse and enabling successful joint learning (Wang et al., 30 Oct 2025).

These variants imply that “co-evolution” spans at least three levels: parameter-level adaptation of a predictive model, state-level growth of an external search structure, and curriculum-level evolution of environments and tasks. PathWise is a useful boundary case because its policy, world model, and critics co-evolve through the changing entailment graph and routed reflections, while model weights themselves remain fixed (Gungordu et al., 28 Jan 2026).

4. Domain instantiations and empirical behavior

The paradigm has been instantiated in embodied planning, Web navigation, tool use, automated heuristic design, GPU-kernel optimization, visual planning, search-agent evaluation, and scalable agent training.

COMAP reports gains across embodied task planning, Web navigation, and tool-use benchmarks, and states that it “consistently outperforms competitive baselines,” including “+16.75% relative improvement with Qwen3-4B.” Its analyses further report that the co-evolutionary loop improves world-model prediction accuracy over time and leads to more effective long-horizon decision-making (Liu et al., 1 Jun 2026).

K-Search evaluates on four complex kernels from FlashInfer—GQA Paged Decode, MLA Paged Prefill, MLA Paged Decode, and FP8 MoE—and reports final speedups over the FlashInfer baseline of 76.0, 57.4, 47.1, and 44.1, respectively, for an average of 56.13. The paper states that this is a “2.10× average improvement over OpenEvolve and 2.21× over ShinkaEvolve,” and that on the hardest MoE kernel it achieves a “14.3× higher speedup vs. OpenEvolve.” On the GPUMode TriMul leaderboard for H100, its Triton solution reaches “1 030 µs,” surpassing the prior best human-tuned CUDA entry at “1 074 µs” (Cao et al., 22 Feb 2026).

WebEvolver studies real-world web environments including Mind2Web-Live, WebVoyager, and GAIA-web. On WebVoyager, zero-shot Llama-3.3-70B is reported at “35.6%,” self-improvement adds “+6%” to approximately “42%,” and adding world-model–synthesized data yields an additional “+4%,” reaching approximately “46%.” In inference-time world-model look-ahead, depth 1 gives approximately “46.2%,” depth 2 approximately “51.4%,” and depth 3 approximately “49.5%,” while intrinsic world-model quality drops with rollout depth (Fang et al., 23 Apr 2025).

CoEx evaluates on ALFWorld, PDDL domains, and Jericho. Reported results include ALFWorld success rates of “ReAct 61.9, ExpeL 64.2, Reflexion 88.1, AdaPlanner 91.8, WALL-E 95.0, CoEx 93.3”; PDDL Gripper success/progress of “ReAct 65.0/89.5, HiAgent 75.0/89.9, CoEx 70.0/94.2”; Blocksworld “50.0/65.0, 50.0/68.3, CoEx 80.0/90.0”; and Jericho average progress “ReAct 36.4→CoEx 55.5” with success “10.0→25.0” (Kim et al., 29 Jul 2025).

PathWise reports that it “converges faster to better heuristics, generalizes across different LLM backbones, and scales to larger problem sizes” in combinatorial optimization. Its significance lies less in a single headline metric than in showing that co-evolving search memory can be encoded as a graph of derivations rather than as a flat population of candidates (Gungordu et al., 28 Jan 2026).

CoLA-World reports improvements in video prediction, real-action adaptation, and downstream visual planning. On RoboDesk visual planning with VP $\pi_{\text{ref}}$ 1, the two-stage pipeline achieves an average success rate of “6.9 %,” while CoLA-World reaches “13.1 %.” On LIBERO and RoboDesk real-action adaptation, it improves PSNR and reduces FVD relative to the two-stage baseline (Wang et al., 30 Oct 2025).

Mind-ParaWorld introduces MPW-Bench, “an interactive benchmark spanning 19 domains with 1,608 instances,” and shows that search agents are strong at evidence synthesis when complete information is available, but are limited by evidence collection and coverage in unfamiliar search environments, unreliable evidence sufficiency judgment, and when-to-stop decisions (Chen et al., 5 Mar 2026).

Agent-World evaluates across “23 challenging agent benchmarks.” On MCP-Mark, BFCL-V4, and $\pi_{\text{ref}}$ 2-Bench, Agent-World-8B reaches “8.9, 51.4, 61.8” and Agent-World-14B reaches “13.3, 55.8, 65.4,” while environment scaling from $\pi_{\text{ref}}$ 3 environments yields average scores of “{18.4%,29.5%,35.8%,38.1%,38.5%}” across four representative domains (Dong et al., 20 Apr 2026).

5. Evaluation criteria, diagnostics, and recurrent bottlenecks

Because the world model is part of the search process rather than merely a simulator, evaluation typically separates process quality from end-task success. COMAP explicitly lists “World-model prediction accuracy (e.g. Delta-F1 on action-induced facts),” “Reliability calibration (how well $\pi_{\text{ref}}$ 4 correlates with actual outcome),” and “End-to-end task success rate” as metrics for search quality. Its reported trend is that “M’s Delta-F1 climbs steadily,” the “adoption rate of student predictions rises,” and “overall task success improves by double-digit percentages” (Liu et al., 1 Jun 2026).

Mind-ParaWorld provides a process-level diagnostic suite tailored to search agents. Final accuracy is measured with Pass@1, while process metrics include Fact Coverage Rate,

$\pi_{\text{ref}}$ 5

Hit Rate,

$\pi_{\text{ref}}$ 6

and ToolCalls. The paper emphasizes that “insufficient coverage is almost always the primary failure mode,” and notes that agents “struggle to adapt queries over time” and to decide “when to stop searching” (Chen et al., 5 Mar 2026).

WebEvolver evaluates world-model fidelity intrinsically using “structural correctness (STR), similarity (Sim.), overall assessment (O/A) as scored by GPT-4o.” At rollout depth 1, reported values are “STR = 75.96%, Sim = 63.56%, O/A = 72.86%”; at depth 2, “O/A ~ 52.32%”; and beyond depth 3, “O/A < 45%.” The search implication is direct: deeper look-ahead increases compounding model error, so inference-time planning quality is bounded by rollout reliability (Fang et al., 23 Apr 2025).

Agent-World evaluates training dynamics under GRPO and reports that reward curves “ascend steadily” while actor entropy increases, which it interprets as continued exploration of new tool-use patterns rather than behavioral collapse. This is relevant because in a co-evolving regime, search degeneration can arise either from an inaccurate world model or from curriculum narrowing; entropy trends provide a partial diagnostic for the latter (Dong et al., 20 Apr 2026).

A recurring issue across the literature is reliability management. COMAP uses teacher–student KL divergence as an explicit confidence proxy; CoEx constrains textual belief updates to verified facts and allows later overwriting of erroneous inferences; WebEvolver tolerates hallucinations during training but relies on an evaluator during look-ahead; CoLA-World inserts a warm-up stage to prevent representational collapse before joint optimization (Liu et al., 1 Jun 2026, Kim et al., 29 Jul 2025, Fang et al., 23 Apr 2025, Wang et al., 30 Oct 2025). This suggests that co-evolution is beneficial only when paired with mechanisms that prevent self-reinforcement of model error.

6. Limitations, misconceptions, and open directions

Several limitations recur across implementations. In WebEvolver, deeper look-ahead does not monotonically improve performance: depth $\pi_{\text{ref}}$ 7 gives the best trade-off, while for $\pi_{\text{ref}}$ 8 the world model becomes less reliable as errors compound. The same paper also reports only limited gains on “Level 2 (deep reasoning + web),” indicating that search quality alone does not resolve harder reasoning bottlenecks (Fang et al., 23 Apr 2025).

CoEx identifies a different failure regime: “abrupt or rapidly shifting environments may require faster belief resets or more aggressive pruning,” and “LLM inference may occasionally hallucinate in $\pi_{\text{ref}}$ 9.” It also notes that scaling to very large tasks may demand memory–belief compression or stronger hierarchy beyond subgoals (Kim et al., 29 Jul 2025).

K-Search highlights computational constraints. Its current world-model updates rely on “vanilla in-context learning,” and the “volume of LLM calls may be a bottleneck”; the paper names “a distilled, lightweight surrogate model for $a_t^i\sim \pi_{\text{draft}}(\cdot\mid s_t)$ 0” and extension to “multi-objective tuning (e.g. latency + energy)” as future directions (Cao et al., 22 Feb 2026).

Mind-ParaWorld frames an evaluative controversy rather than a deployment limitation. Static search benchmarks can suffer “difficulty collapse,” “fact drift,” and “confounded attribution,” making it hard to determine whether a model truly searched or simply exploited parametric memory. Its controlled parallel-world construction is therefore best understood as a response to a measurement problem in search-agent research rather than as a direct search algorithm (Chen et al., 5 Mar 2026).

Agent-World shows diminishing returns from environment scaling and from repeated self-evolution rounds: performance rises rapidly from 10 to 500 environments and then flattens, while “two rounds remain positive but smaller” than the first. This indicates that environment co-evolution is not unbounded and may require better diagnosis or diversification strategies at larger scales (Dong et al., 20 Apr 2026).

A final misconception is to equate co-evolution with parameter updating alone. PathWise demonstrates that critic feedback can condition future policy and world-model calls while “never” updating model weights, with only the graph state $a_t^i\sim \pi_{\text{draft}}(\cdot\mid s_t)$ 1 changing permanently (Gungordu et al., 28 Jan 2026). Conversely, CoLA-World shows the opposite extreme, where co-evolution occurs through fully joint optimization of a latent action model and a pre-trained world model after a carefully staged warm-up (Wang et al., 30 Oct 2025). The literature therefore supports a broader definition: search via a co-evolving world model is any search regime in which the model of future possibilities is revised by the very process of searching, whether through weight updates, state-graph growth, belief revision, or environment synthesis.