Failure Trajectory Closed-Loop Evolver

Updated 4 July 2026

Failure Trajectory Closed-Loop Evolver is a framework that repurposes failed trajectories into supervisory signals for continuous system improvement.
It operates by collecting trajectories, identifying failure patterns, generating repair candidates, and validating updates in an iterative feedback loop.
This approach is applied across domains such as autonomous driving, LLM agents, and robotics, yielding significant gains in robustness and safety.

A Failure Trajectory Closed-Loop Evolver is a closed-loop learning mechanism in which failed trajectories are treated as reusable supervision rather than terminal errors. Across recent systems, the object being evolved may be a skill library, a driving policy, a world model, a diagnostic planner, or an agent policy, but the recurring pattern is the same: collect trajectories, identify failure structure, generate repairs or adversarial variants, validate them, and feed the result back into the next interaction or training round. The term is explicit in AOI, which “mines unsuccessful trajectories and converts them into corrective supervision signals” (Yang et al., 3 Mar 2026), and closely related realizations appear in SkillClaw’s collective skill evolution (Ma et al., 9 Apr 2026), E²’s adversarial curriculum for autonomous vehicles (Guo et al., 8 Apr 2026), EvolveR’s experience-driven lifecycle (Wu et al., 17 Oct 2025), World-VLA-Loop’s co-evolving world-model/policy loop (Liu et al., 6 Feb 2026), MPA’s model-based policy adaptation (Lin et al., 26 Nov 2025), KG-ASG’s collision-knowledge-guided closed-loop generation (Wang et al., 17 May 2026), and FATE’s on-policy self-evolution for agentic safety alignment (Yin et al., 12 May 2026).

1. Conceptual structure

The defining property of the evolver is closure of the feedback loop. In conventional open-loop pipelines, failures are discovered after training or deployment and rarely change the system that produced them. E² states this contrast directly: conventional pipelines train an ego policy and then perform a separate, open-loop adversarial evaluation step, whereas E² closes the loop by generating failure-mode trajectories conditioned on the current policy, simulating them, and recycling the resulting failures as training data (Guo et al., 8 Apr 2026). SkillClaw makes a parallel claim for multi-user LLM agents: interactions from different users provide complementary signals about when a skill works or fails, but existing systems lack a mechanism to convert such heterogeneous experiences into reliable skill updates; its framework continuously aggregates trajectories and uses an autonomous evolver to refine or extend skills (Ma et al., 9 Apr 2026).

The architectural locus of evolution differs by domain. In AOI, the evolved artifact is a corrected command sequence returned as a structured prompt, “Corrected Diagnostic Plan,” which is then consumed by the Observer while maintaining read-write separation (Yang et al., 3 Mar 2026). In FATE, the evolved artifact is the policy itself: on-policy failures are repaired, filtered by verifier scores, and replayed through supervised fine-tuning and Pareto-Front Policy Optimization (PFPO) (Yin et al., 12 May 2026). In World-VLA-Loop, failures generated by the current VLA policy refine the video world model, and the refined world model then improves subsequent RL optimization (Liu et al., 6 Feb 2026). In MPA, the evolver acts at inference time by proposing multiple adapted trajectories and executing the one with the highest estimated long-horizon utility (Lin et al., 26 Nov 2025). This suggests that the phrase names an architectural pattern rather than a single algorithm.

A second recurring property is that the evolver operates on trajectory-level structure rather than only final outcomes. FATE motivates this explicitly by noting that tool-using agents fail through trajectories rather than only final responses (Yin et al., 12 May 2026). AOI likewise assumes that failed diagnostic traces contain “near-miss” information such as the correct target component but a wrong flag (Yang et al., 3 Mar 2026). SkillClaw treats the full causal chain plus metadata as structured evidence, while successful trajectories act as “invariants” that must not be broken (Ma et al., 9 Apr 2026).

2. Formal representations of failure trajectories

The trajectory object is domain-specific, but recent systems converge on explicit state–action–feedback encodings.

System	Trajectory form	Failure annotation
SkillClaw (Ma et al., 9 Apr 2026)	$\tau = [(u_0,a_1,f_1), \ldots, (u_{T-1},a_T,f_T)]$	$e(\tau)=1$ or low $q(\tau)$
EvolveR (Wu et al., 17 Oct 2025)	$\tau = \{(s_1,a_1,r_1), \ldots, (s_T,a_T,r_T)\}$	$y(\tau)=0$
AOI (Yang et al., 3 Mar 2026)	$\tau^-=(c_1,r_1,c_2,r_2,\ldots,c_T,r_T)$	unsuccessful diagnostic trajectory
FATE (Yin et al., 12 May 2026)	$f=(x,a,z(x,a))$	at least one verifier score violates threshold
E² (Guo et al., 8 Apr 2026)	joint-agent path $\tau$ under $p_u(\tau)$	terminal failure criterion such as collision
World-VLA-Loop (Liu et al., 6 Feb 2026)	rollout buffer entries $(x_t,a_t,r_t)$	final $e(\tau)=1$ 0

SkillClaw records each independent user–agent session as $e(\tau)=1$ 1, where $e(\tau)=1$ 2 is the user or system prompt, $e(\tau)=1$ 3 is the agent’s action, and $e(\tau)=1$ 4 is the immediate feedback. It further extracts lightweight metadata: $e(\tau)=1$ 5 for invoked skills, $e(\tau)=1$ 6 for whether any tool error occurred, and $e(\tau)=1$ 7 as a coarse quality estimate. Trajectories are collected into a central store $e(\tau)=1$ 8, lifted into evidence records $e(\tau)=1$ 9, and grouped per skill as $q(\tau)$ 0, with a special no-skill group $q(\tau)$ 1 (Ma et al., 9 Apr 2026).

EvolveR uses the canonical RL-style trajectory $q(\tau)$ 2 and attaches a binary success flag $q(\tau)$ 3. This induces the partition $q(\tau)$ 4 and $q(\tau)$ 5, which is central to its later distinction between Guiding Principles and Cautionary Principles (Wu et al., 17 Oct 2025). AOI makes the command–response alternation explicit: a failed diagnostic trajectory is $q(\tau)$ 6, where $q(\tau)$ 7 is a command issued by the Observer/Executor and $q(\tau)$ 8 is the raw response (Yang et al., 3 Mar 2026).

FATE adds a multi-objective verifier signature to the trajectory. A failure is $q(\tau)$ 9, where $\tau = \{(s_1,a_1,r_1), \ldots, (s_T,a_T,r_T)\}$ 0 is the task or environment state, $\tau = \{(s_1,a_1,r_1), \ldots, (s_T,a_T,r_T)\}$ 1 is the generated trajectory, and $\tau = \{(s_1,a_1,r_1), \ldots, (s_T,a_T,r_T)\}$ 2 measures security, task utility, over-refusal control, and trajectory control (Yin et al., 12 May 2026). In E², the object is a controlled trajectory under a reverse-time SDE, and failure is encoded in the terminal cost $\tau = \{(s_1,a_1,r_1), \ldots, (s_T,a_T,r_T)\}$ 3, such as a collision indicator or collision severity (Guo et al., 8 Apr 2026). World-VLA-Loop labels a rollout as SUCCESS if $\tau = \{(s_1,a_1,r_1), \ldots, (s_T,a_T,r_T)\}$ 4 and FAILURE otherwise, then collects the failure set as $\tau = \{(s_1,a_1,r_1), \ldots, (s_T,a_T,r_T)\}$ 5 (Liu et al., 6 Feb 2026).

3. Closed-loop operators and update targets

Taken together, the cited systems suggest a common operator sequence: failure mining, candidate generation, candidate selection, and artifact update.

In SkillClaw, the collected sessions are first passed through structure_and_annotate(T). For each skill $\tau = \{(s_1,a_1,r_1), \ldots, (s_T,a_T,r_T)\}$ 6, the AgenticEvolver reads the corresponding evidence group $\tau = \{(s_1,a_1,r_1), \ldots, (s_T,a_T,r_T)\}$ 7, decides whether to refine or create, generates a candidate update, and passes it to a Validator that replays a small set of representative tasks before replacing or adding skills. The same mechanism is applied to recurring patterns in the no-skill group $\tau = \{(s_1,a_1,r_1), \ldots, (s_T,a_T,r_T)\}$ 8, enabling new skill creation from repeated unsupported procedures (Ma et al., 9 Apr 2026).

AOI implements repair generation as a sequence-to-sequence policy $\tau = \{(s_1,a_1,r_1), \ldots, (s_T,a_T,r_T)\}$ 9 fine-tuned with GRPO. Given a failed trace, the Evolver samples $y(\tau)=0$ 0 candidate corrected plans, scores each with a reward model $y(\tau)=0$ 1, and emits the highest-scoring candidate as a structured prompt. The system then restarts a fresh diagnostic attempt, seeding the Observer with that plan while keeping the Evolver offline and non-executing (Yang et al., 3 Mar 2026).

FATE makes the loop explicitly on-policy. At each round, the current policy $y(\tau)=0$ 2 is rolled out on development tasks to build a failure set $y(\tau)=0$ 3. For each failure, the same policy samples $y(\tau)=0$ 4 repair candidates conditioned on a repair prompt containing the failed trajectory and verifier feedback; the candidates are re-scored, filtered by protected-objective thresholds, projected onto a Pareto front, and added to a replay buffer. The next policy is obtained by supervised warmup followed by PFPO refinement (Yin et al., 12 May 2026).

E² and KG-ASG instantiate the evolver in autonomous driving through adversarial scenario generation rather than repaired text plans. E² samples scenes from a buffer, generates a nominal rollout under $y(\tau)=0$ 5, selects adversarial support agents, injects a topological anchor into a controlled reverse SDE, rolls the simulator forward from the adversarial initial state, and updates the policy with the resulting failures (Guo et al., 8 Apr 2026). KG-ASG generates scenarios through a primary-support process guided by a Collision Expert, then executes the scenario under a planner-controller to obtain feedback $y(\tau)=0$ 6; if no collision occurs, it updates the ego reference trajectory and retry profile for the next round, up to $y(\tau)=0$ 7, with terminal refinement afterward (Wang et al., 17 May 2026).

World-VLA-Loop changes the update target again. The loop first simulates rollouts inside the current world model, extracts failure trajectories, augments the SANS dataset with those failures, updates the world model, and only then performs RL post-training of the VLA policy in the refined simulator (Liu et al., 6 Feb 2026). MPA separates data generation, offline adaptation, and inference-time evolution: counterfactual rollouts collected from a geometry-consistent simulator train both a diffusion-based policy adapter and a multi-step Q-value model, and closed-loop execution then selects the best proposal among sampled candidates (Lin et al., 26 Nov 2025).

4. Objectives, losses, and scoring functions

One of the clearest distinctions among failure-trajectory evolvers is whether they rely on an explicit optimization objective or on operational acceptance tests.

SkillClaw explicitly does not posit a differentiable loss or explicit weighting formula. Its objective is operational: identify updates $y(\tau)=0$ 8 to the skill set that increase the expected task success rate over the next round of interactions. Failures, defined by $y(\tau)=0$ 9 or low $\tau^-=(c_1,r_1,c_2,r_2,\ldots,c_T,r_T)$ 0, are high-signal and focus the evolver on broken procedural steps, while successful trajectories are treated as invariants. Candidate revisions are accepted if

$\tau^-=(c_1,r_1,c_2,r_2,\ldots,c_T,r_T)$ 1

as estimated by replaying a small validation pool under both versions (Ma et al., 9 Apr 2026).

By contrast, E² formulates adversarial synthesis as transport-regularized control over a reverse-time SDE. The controlled process

$\tau^-=(c_1,r_1,c_2,r_2,\ldots,c_T,r_T)$ 2

induces a path measure $\tau^-=(c_1,r_1,c_2,r_2,\ldots,c_T,r_T)$ 3, and the control is chosen to induce failures while penalizing deviation from the nominal distribution through a KL term:

$\tau^-=(c_1,r_1,c_2,r_2,\ldots,c_T,r_T)$ 4

Using Girsanov’s theorem, the KL term becomes an explicit quadratic control cost (Guo et al., 8 Apr 2026).

Several agentic systems use multi-stage scoring and policy optimization. EvolveR defines a distillation loss

$\tau^-=(c_1,r_1,c_2,r_2,\ldots,c_T,r_T)$ 5

with $\tau^-=(c_1,r_1,c_2,r_2,\ldots,c_T,r_T)$ 6 for failures chosen larger than $\tau^-=(c_1,r_1,c_2,r_2,\ldots,c_T,r_T)$ 7 for successes so that cautionary principles receive extra representation (Wu et al., 17 Oct 2025). AOI uses GRPO with group-normalized advantage

$\tau^-=(c_1,r_1,c_2,r_2,\ldots,c_T,r_T)$ 8

and a reward model composed of Validity, Completeness, Correctness, and Effectiveness sub-scores (Yang et al., 3 Mar 2026). FATE first builds a supervised replay buffer through Pareto-front filtering and then optimizes a clipped PPO-style objective with KL regularization to a frozen reference policy, using front-only reward $\tau^-=(c_1,r_1,c_2,r_2,\ldots,c_T,r_T)$ 9 over security, utility, over-refusal, and control (Yin et al., 12 May 2026).

World-VLA-Loop couples world-model fitting with policy optimization. The world model is refined on an augmented SANS dataset by minimizing a sum of diffusion flow-matching loss and reward-prediction regression, while the policy is updated by policy gradient on simulated trajectories; the paper notes that GRPO can be used in practice (Liu et al., 6 Feb 2026). MPA trains a diffusion residual adapter with denoising score matching and a multi-step Q-value model with return regression,

$f=(x,a,z(x,a))$ 0

then uses the Q model as an inference-time selector over sampled candidates (Lin et al., 26 Nov 2025).

5. Failure attribution, safeguards, and common misconceptions

A common misconception is that failure-trajectory evolution is merely aggressive hard-negative mining. The systems surveyed here are more constrained: they generally combine failure discovery with attribution, realism control, and deployment safeguards.

SkillClaw triggers inspection only when recurring failure patterns appear, using empirical failure rate $f=(x,a,z(x,a))$ 1 and minimum-support conditions, and it validates edits through replay before synchronizing them system-wide (Ma et al., 9 Apr 2026). AOI enforces read-write separation: the Evolver only reads failed traces and never interacts with the real cluster; corrected plans are text prompts, while actual execution remains under Probe/Executor with whitelists (Yang et al., 3 Mar 2026). FATE similarly does not accept arbitrary repairs: candidates must satisfy task-mode-specific thresholds $f=(x,a,z(x,a))$ 2 for utility, over-refusal, and control, and only non-dominated feasible candidates on the Pareto front can enter the replay buffer (Yin et al., 12 May 2026).

In autonomous driving, safeguard design is especially explicit. E² uses topology-driven support selection to avoid perturbing all agents, semantic feasibility filtering to enforce map consistency and route conflict relevance, and Topological Anchoring to stabilize rare failure modes while constraining deviations from realistic data distributions (Guo et al., 8 Apr 2026). KG-ASG imposes hard-gate constraints

$f=(x,a,z(x,a))$ 3

together with a single-collider attribution rule requiring that support vehicles not collide with the ego and that the number of colliders be at most one (Wang et al., 17 May 2026). MPA prunes trajectories that stray beyond a lateral threshold $f=(x,a,z(x,a))$ 4 or incur low heuristic reward in order to maintain visual fidelity (Lin et al., 26 Nov 2025).

Another misconception is that these systems always update the policy directly. The update target is often factorized. EvolveR distills failures into cautionary principles stored in a repository $f=(x,a,z(x,a))$ 5 and retrieved later through cosine-based scoring and dynamic utility score $f=(x,a,z(x,a))$ 6 (Wu et al., 17 Oct 2025). World-VLA-Loop first improves simulator fidelity on policy-induced failures, then exploits the improved simulator for RL (Liu et al., 6 Feb 2026). This suggests that “evolution” may occur at the level of memory, skill definitions, scenario generators, simulators, or proposal distributions rather than only model weights.

6. Empirical behavior and research significance

Reported empirical results indicate that closed-loop failure reuse can improve both failure discovery and downstream robustness, although the metrics differ sharply across domains.

System	Reported effect
SkillClaw (Ma et al., 9 Apr 2026)	By Day 6 vs Day 1: +11.7% Social Interaction, +52.0% Search & Retrieval, +88.4% Creative Synthesis, +33.3% Safety Alignment
E² (Guo et al., 8 Apr 2026)	CFR 60.29% vs 51.28% on nuScenes; 78.57% vs 57.14% on nuPlan
AOI (Yang et al., 3 Mar 2026)	Evolver converts 37 failed trajectories into diagnostic guidance; avg@5 improves by 4.8 points; variance reduced by 35%
KG-ASG (Wang et al., 17 May 2026)	ASR $f=(x,a,z(x,a))$ 7; Valid Primary Attack 92.2%; Multi-collision 0.00%
World-VLA-Loop (Liu et al., 6 Feb 2026)	After two iterations, real-world success rate of OpenVLA-OFT jumps by +36.7%; held-out failure/success matching reaches $f=(x,a,z(x,a))$ 8
MPA (Lin et al., 26 Nov 2025)	RC rises from $f=(x,a,z(x,a))$ 9– $\tau$ 0 to $\tau$ 1– $\tau$ 2; HDScore from $\tau$ 3– $\tau$ 4 to $\tau$ 5– $\tau$ 6
FATE (Yin et al., 12 May 2026)	Attack success rate reduced by 33.5%; harmful compliance reduced by 82.6%; external trajectory-safety diagnosis improves by 6.5%

Within individual studies, the results are tied to the structure of the evolver. SkillClaw’s deployment simulation on WildClawBench uses six “days” with daytime collection by 8 concurrent users and nighttime skill evolution, and also reports a “Skill Evolve Lite” average gain of +42.1% after a single evolution pass on 3 custom queries (Ma et al., 9 Apr 2026). E² not only improves collision failure discovery but also reports robustness gains after recycling generated failures for fine-tuning: on a held-out nuScenes split under matched intensity $\tau$ 7, failure rates drop by 100%, while under higher intensity $\tau$ 8 they still decrease by $\tau$ 9– $p_u(\tau)$ 0 (Guo et al., 8 Apr 2026). AOI reports that all 37 failed test cases were converted into corrected plans, with LLM-judge mean repair quality increasing from 7.18 to 8.27 and standard deviation dropping from 0.97 to 0.49 (Yang et al., 3 Mar 2026). FATE reports progressive reductions in unsafe rates across evolution rounds on AgentDojo and AgentHarm while preserving or improving task success and refusal calibration (Yin et al., 12 May 2026).

A plausible implication is that failure trajectories provide a denser supervisory object than terminal labels because they retain causal sequencing: command–response alternation in AOI, verifier dimensions in FATE, skill references and error flags in SkillClaw, topological interactions in E² and KG-ASG, and action-conditioned visual consequences in World-VLA-Loop. The recent literature therefore portrays the Failure Trajectory Closed-Loop Evolver as a unifying systems pattern for continual improvement under deployment feedback, with concrete instantiations spanning LLM agents, autonomous driving, cloud operations, and robotics.