---
title: Failure Trajectory Closed-Loop Evolver
url: https://www.emergentmind.com/topics/failure-trajectory-closed-loop-evolver
type: topic
---

# Failure Trajectory Closed-Loop Evolver

A **Failure Trajectory Closed-Loop Evolver** is a closed-loop learning mechanism in which failed trajectories are treated as reusable supervision rather than terminal errors. Across recent systems, the object being evolved may be a skill library, a driving policy, a world model, a diagnostic planner, or an agent policy, but the recurring pattern is the same: collect trajectories, identify failure structure, generate repairs or adversarial variants, validate them, and feed the result back into the next interaction or training round. The term is explicit in AOI, which “mines unsuccessful trajectories and converts them into corrective supervision signals” [2603.03378], and closely related realizations appear in SkillClaw’s collective skill evolution [2604.08377], E²’s adversarial curriculum for autonomous vehicles [2604.07378], EvolveR’s experience-driven lifecycle [2510.16079], World-VLA-Loop’s co-evolving world-model/policy loop [2602.06508], MPA’s model-based policy adaptation [2511.21584], KG-ASG’s collision-knowledge-guided closed-loop generation [2605.18895], and FATE’s on-policy self-evolution for agentic safety alignment [2605.11882].

## 1. Conceptual structure

The defining property of the evolver is **closure of the feedback loop**. In conventional open-loop pipelines, failures are discovered after training or deployment and rarely change the system that produced them. E² states this contrast directly: conventional pipelines train an ego policy and then perform a separate, open-loop adversarial evaluation step, whereas E² closes the loop by generating failure-mode trajectories conditioned on the current policy, simulating them, and recycling the resulting failures as training data [2604.07378]. SkillClaw makes a parallel claim for multi-user LLM agents: interactions from different users provide complementary signals about when a skill works or fails, but existing systems lack a mechanism to convert such heterogeneous experiences into reliable skill updates; its framework continuously aggregates trajectories and uses an autonomous evolver to refine or extend skills [2604.08377].

The architectural locus of evolution differs by domain. In AOI, the evolved artifact is a corrected command sequence returned as a structured prompt, “Corrected Diagnostic Plan,” which is then consumed by the Observer while maintaining read-write separation [2603.03378]. In FATE, the evolved artifact is the policy itself: on-policy failures are repaired, filtered by verifier scores, and replayed through supervised fine-tuning and Pareto-Front Policy Optimization (PFPO) [2605.11882]. In World-VLA-Loop, failures generated by the current VLA policy refine the video world model, and the refined world model then improves subsequent RL optimization [2602.06508]. In MPA, the evolver acts at inference time by proposing multiple adapted trajectories and executing the one with the highest estimated long-horizon utility [2511.21584]. This suggests that the phrase names an architectural pattern rather than a single algorithm.

A second recurring property is that the evolver operates on **trajectory-level structure** rather than only final outcomes. FATE motivates this explicitly by noting that tool-using agents fail through trajectories rather than only final responses [2605.11882]. AOI likewise assumes that failed diagnostic traces contain “near-miss” information such as the correct target component but a wrong flag [2603.03378]. SkillClaw treats the full causal chain plus metadata as structured evidence, while successful trajectories act as “invariants” that must not be broken [2604.08377].

## 2. Formal representations of failure trajectories

The trajectory object is domain-specific, but recent systems converge on explicit state–action–feedback encodings.

| System | Trajectory form | Failure annotation |
|---|---|---|
| SkillClaw [2604.08377] | $\tau = [(u_0,a_1,f_1), \ldots, (u_{T-1},a_T,f_T)]$ | $e(\tau)=1$ or low $q(\tau)$ |
| EvolveR [2510.16079] | $\tau = \{(s_1,a_1,r_1), \ldots, (s_T,a_T,r_T)\}$ | $y(\tau)=0$ |
| AOI [2603.03378] | $\tau^-=(c_1,r_1,c_2,r_2,\ldots,c_T,r_T)$ | unsuccessful diagnostic trajectory |
| FATE [2605.11882] | $f=(x,a,z(x,a))$ | at least one verifier score violates threshold |
| E² [2604.07378] | joint-agent path $\tau$ under $p_u(\tau)$ | terminal failure criterion such as collision |
| World-VLA-Loop [2602.06508] | rollout buffer entries $(x_t,a_t,r_t)$ | final $R(\tau)<\delta$ |

SkillClaw records each independent user–agent session as $\tau = [(u_0, a_1, f_1), \ldots, (u_{T-1}, a_T, f_T)]$, where $u_t$ is the user or system prompt, $a_t$ is the agent’s action, and $f_t$ is the immediate feedback. It further extracts lightweight metadata: $\mathcal{K}(\tau)\subseteq\mathcal{S}$ for invoked skills, $e(\tau)\in\{0,1\}$ for whether any tool error occurred, and $q(\tau)\in[0,1]$ as a coarse quality estimate. Trajectories are collected into a central store $\mathcal{T}$, lifted into evidence records $\mathcal{E}$, and grouped per skill as $G(s)=\{\tau\in\mathcal{T}\mid s\in\mathcal{K}(\tau)\}$, with a special no-skill group $G(\emptyset)$ [2604.08377].

EvolveR uses the canonical RL-style trajectory $\tau=\{(s_t,a_t,r_t)\}_{t=1}^T$ and attaches a binary success flag $y(\tau)$. This induces the partition $\mathcal{D}_+=\{\tau\mid y(\tau)=1\}$ and $\mathcal{D}_-=\{\tau\mid y(\tau)=0\}$, which is central to its later distinction between Guiding Principles and Cautionary Principles [2510.16079]. AOI makes the command–response alternation explicit: a failed diagnostic trajectory is $\tau^-=(c_1,r_1,\ldots,c_T,r_T)$, where $c_i$ is a command issued by the Observer/Executor and $r_i$ is the raw response [2603.03378].

FATE adds a multi-objective verifier signature to the trajectory. A failure is $f=(x,a,z(x,a))$, where $x$ is the task or environment state, $a$ is the generated trajectory, and $z(x,a)=(z_{\text{sec}}, z_{\text{util}}, z_{\text{or}}, z_{\text{ctrl}})\in[0,1]^4$ measures security, task utility, over-refusal control, and trajectory control [2605.11882]. In E², the object is a controlled trajectory under a reverse-time SDE, and failure is encoded in the terminal cost $C(x_T)$, such as a collision indicator or collision severity [2604.07378]. World-VLA-Loop labels a rollout as SUCCESS if $R(\tau)\ge\delta$ and FAILURE otherwise, then collects the failure set as $F_{\text{fail}}=\{\tau\in D_{\text{sim}}\mid \text{final }R(\tau)<\delta\}$ [2602.06508].

## 3. Closed-loop operators and update targets

Taken together, the cited systems suggest a common operator sequence: **failure mining, candidate generation, candidate selection, and artifact update**.

In SkillClaw, the collected sessions are first passed through `structure_and_annotate(T)`. For each skill $s$, the AgenticEvolver reads the corresponding evidence group $G_s$, decides whether to refine or create, generates a candidate update, and passes it to a Validator that replays a small set of representative tasks before replacing or adding skills. The same mechanism is applied to recurring patterns in the no-skill group $G_\emptyset$, enabling new skill creation from repeated unsupported procedures [2604.08377].

AOI implements repair generation as a sequence-to-sequence policy $\pi_e(\cdot\mid \tau,\text{problem})$ fine-tuned with GRPO. Given a failed trace, the Evolver samples $G$ candidate corrected plans, scores each with a reward model $R_{\text{corr}}(\tau^-,\hat\tau)\in[0,1]$, and emits the highest-scoring candidate as a structured prompt. The system then restarts a fresh diagnostic attempt, seeding the Observer with that plan while keeping the Evolver offline and non-executing [2603.03378].

FATE makes the loop explicitly on-policy. At each round, the current policy $\pi_t$ is rolled out on development tasks to build a failure set $F_t$. For each failure, the same policy samples $K$ repair candidates conditioned on a repair prompt containing the failed trajectory and verifier feedback; the candidates are re-scored, filtered by protected-objective thresholds, projected onto a Pareto front, and added to a replay buffer. The next policy is obtained by supervised warmup followed by PFPO refinement [2605.11882].

E² and KG-ASG instantiate the evolver in autonomous driving through adversarial scenario generation rather than repaired text plans. E² samples scenes from a buffer, generates a nominal rollout under $\pi_{\theta^{(r)}}$, selects adversarial support agents, injects a topological anchor into a controlled reverse SDE, rolls the simulator forward from the adversarial initial state, and updates the policy with the resulting failures [2604.07378]. KG-ASG generates scenarios through a primary-support process guided by a Collision Expert, then executes the scenario under a planner-controller to obtain feedback $h^{(r)}=(\xi_0^{(r)}, I_{\text{col}}^{(r)}, d_{\min}^{(r)}, e^{(r)})$; if no collision occurs, it updates the ego reference trajectory and retry profile for the next round, up to $R_{\max}=5$, with terminal refinement afterward [2605.18895].

World-VLA-Loop changes the update target again. The loop first simulates rollouts inside the current world model, extracts failure trajectories, augments the SANS dataset with those failures, updates the world model, and only then performs RL post-training of the VLA policy in the refined simulator [2602.06508]. MPA separates data generation, offline adaptation, and inference-time evolution: counterfactual rollouts collected from a geometry-consistent simulator train both a diffusion-based policy adapter and a multi-step Q-value model, and closed-loop execution then selects the best proposal among sampled candidates [2511.21584].

## 4. Objectives, losses, and scoring functions

One of the clearest distinctions among failure-trajectory evolvers is whether they rely on an explicit optimization objective or on operational acceptance tests.

SkillClaw explicitly does **not** posit a differentiable loss or explicit weighting formula. Its objective is operational: identify updates $\Delta$ to the skill set that increase the expected task success rate over the next round of interactions. Failures, defined by $e(\tau)=1$ or low $q(\tau)$, are high-signal and focus the evolver on broken procedural steps, while successful trajectories are treated as invariants. Candidate revisions are accepted if
$$
\Delta\mathrm{Perf}(s')=\mathbb{E}_{\tau\sim G(s)}[\mathrm{success}(s',\tau)]-\mathbb{E}_{\tau\sim G(s)}[\mathrm{success}(s,\tau)]>0
$$
as estimated by replaying a small validation pool under both versions [2604.08377].

By contrast, E² formulates adversarial synthesis as transport-regularized control over a reverse-time SDE. The controlled process
$$
d x_t = [f(x_t,t)-g(t)^2 s_\phi(x_t,t)+u_t]dt + g(t)d\bar w_t
$$
induces a path measure $p_u(\tau)$, and the control is chosen to induce failures while penalizing deviation from the nominal distribution through a KL term:
$$
\min_{u_t}\; \mathbb{E}_{\tau\sim p_u}\!\left[\int_0^T R(u_t)\,dt + C(x_T)\right] + \lambda D_{KL}(p_u(\tau)\|p_0(\tau)).
$$
Using Girsanov’s theorem, the KL term becomes an explicit quadratic control cost [2604.07378].

Several agentic systems use multi-stage scoring and policy optimization. EvolveR defines a distillation loss
$$
L_{\mathrm{distill}}(\phi)=\sum_{\tau\in\mathcal{D}_{\mathrm{new}}} w(\tau)\;\mathrm{CE}(f_\phi(\tau),p_{\mathrm{cand}}(\tau)),
$$
with $w_-=w(\tau)$ for failures chosen larger than $w_+=1.0$ for successes so that cautionary principles receive extra representation [2510.16079]. AOI uses GRPO with group-normalized advantage
$$
A_i=\frac{R(\tau^-,\tau_i^+) - \mu_G}{\sigma_G+\epsilon}
$$
and a reward model composed of Validity, Completeness, Correctness, and Effectiveness sub-scores [2603.03378]. FATE first builds a supervised replay buffer through Pareto-front filtering and then optimizes a clipped PPO-style objective with KL regularization to a frozen reference policy, using front-only reward $r_{PF}$ over security, utility, over-refusal, and control [2605.11882].

World-VLA-Loop couples world-model fitting with policy optimization. The world model is refined on an augmented SANS dataset by minimizing a sum of diffusion flow-matching loss and reward-prediction regression, while the policy is updated by policy gradient on simulated trajectories; the paper notes that GRPO can be used in practice [2602.06508]. MPA trains a diffusion residual adapter with denoising score matching and a multi-step Q-value model with return regression,
$$
L_Q=\mathbb{E}_{(o,s,a)\in D_{cf}}\left[\left(Q(o,s,a)-\sum_{k=0}^T \gamma^k r_{t+k}\right)^2\right],
$$
then uses the Q model as an inference-time selector over sampled candidates [2511.21584].

## 5. Failure attribution, safeguards, and common misconceptions

A common misconception is that failure-trajectory evolution is merely aggressive hard-negative mining. The systems surveyed here are more constrained: they generally combine failure discovery with attribution, realism control, and deployment safeguards.

SkillClaw triggers inspection only when recurring failure patterns appear, using empirical failure rate $f_s=\mathrm{fail\_count}(s)/n_s$ and minimum-support conditions, and it validates edits through replay before synchronizing them system-wide [2604.08377]. AOI enforces **read-write separation**: the Evolver only reads failed traces and never interacts with the real cluster; corrected plans are text prompts, while actual execution remains under Probe/Executor with whitelists [2603.03378]. FATE similarly does not accept arbitrary repairs: candidates must satisfy task-mode-specific thresholds $\kappa_\tau$ for utility, over-refusal, and control, and only non-dominated feasible candidates on the Pareto front can enter the replay buffer [2605.11882].

In autonomous driving, safeguard design is especially explicit. E² uses topology-driven support selection to avoid perturbing all agents, semantic feasibility filtering to enforce map consistency and route conflict relevance, and Topological Anchoring to stabilize rare failure modes while constraining deviations from realistic data distributions [2604.07378]. KG-ASG imposes hard-gate constraints
$$
C(\tau)=C_{\text{road}}(\tau)\wedge C_{\text{signal}}(\tau)\wedge C_{\text{dyn}}(\tau)\wedge C_{\text{int}}(\tau),
$$
together with a single-collider attribution rule requiring that support vehicles not collide with the ego and that the number of colliders be at most one [2605.18895]. MPA prunes trajectories that stray beyond a lateral threshold $\delta$ or incur low heuristic reward in order to maintain visual fidelity [2511.21584].

Another misconception is that these systems always update the policy directly. The update target is often **factorized**. EvolveR distills failures into cautionary principles stored in a repository $\mathcal{E}$ and retrieved later through cosine-based scoring and dynamic utility score $s(p)$ [2510.16079]. World-VLA-Loop first improves simulator fidelity on policy-induced failures, then exploits the improved simulator for RL [2602.06508]. This suggests that “evolution” may occur at the level of memory, skill definitions, scenario generators, simulators, or proposal distributions rather than only model weights.

## 6. Empirical behavior and research significance

Reported empirical results indicate that closed-loop failure reuse can improve both failure discovery and downstream robustness, although the metrics differ sharply across domains.

| System | Reported effect |
|---|---|
| SkillClaw [2604.08377] | By Day 6 vs Day 1: +11.7% Social Interaction, +52.0% Search & Retrieval, +88.4% Creative Synthesis, +33.3% Safety Alignment |
| E² [2604.07378] | CFR 60.29% vs 51.28% on nuScenes; 78.57% vs 57.14% on nuPlan |
| AOI [2603.03378] | Evolver converts 37 failed trajectories into diagnostic guidance; avg@5 improves by 4.8 points; variance reduced by 35% |
| KG-ASG [2605.18895] | ASR $\approx 92.6\%$; Valid Primary Attack 92.2%; Multi-collision 0.00% |
| World-VLA-Loop [2602.06508] | After two iterations, real-world success rate of OpenVLA-OFT jumps by +36.7%; held-out failure/success matching reaches $\approx 90\%$ |
| MPA [2511.21584] | RC rises from $\sim 40$–$65\%$ to $\sim 90$–$95\%$; HDScore from $\sim 20$–$45$ to $\sim 60$–$75$ |
| FATE [2605.11882] | Attack success rate reduced by 33.5%; harmful compliance reduced by 82.6%; external trajectory-safety diagnosis improves by 6.5% |

Within individual studies, the results are tied to the structure of the evolver. SkillClaw’s deployment simulation on WildClawBench uses six “days” with daytime collection by 8 concurrent users and nighttime skill evolution, and also reports a “Skill Evolve Lite” average gain of +42.1% after a single evolution pass on 3 custom queries [2604.08377]. E² not only improves collision failure discovery but also reports robustness gains after recycling generated failures for fine-tuning: on a held-out nuScenes split under matched intensity $\eta=1.0$, failure rates drop by 100%, while under higher intensity $\eta=2.0$ they still decrease by $\sim 30$–$40\%$ [2604.07378]. AOI reports that all 37 failed test cases were converted into corrected plans, with LLM-judge mean repair quality increasing from 7.18 to 8.27 and standard deviation dropping from 0.97 to 0.49 [2603.03378]. FATE reports progressive reductions in unsafe rates across evolution rounds on AgentDojo and AgentHarm while preserving or improving task success and refusal calibration [2605.11882].

A plausible implication is that failure trajectories provide a denser supervisory object than terminal labels because they retain causal sequencing: command–response alternation in AOI, verifier dimensions in FATE, skill references and error flags in SkillClaw, topological interactions in E² and KG-ASG, and action-conditioned visual consequences in World-VLA-Loop. The recent literature therefore portrays the Failure Trajectory Closed-Loop Evolver as a unifying systems pattern for continual improvement under deployment feedback, with concrete instantiations spanning LLM agents, autonomous driving, cloud operations, and robotics.

Source: https://www.emergentmind.com/topics/failure-trajectory-closed-loop-evolver