Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills

Abstract: Equipping LLM agents with domain-specific skills is critical for tackling complex tasks. Yet, manual authoring creates a severe scalability bottleneck. Conversely, automated skill generation often yields fragile or fragmented results because it either relies on shallow parametric knowledge or sequentially overfits to non-generalizable trajectory-local lessons. To overcome this, we introduce Trace2Skill, a framework that mirrors how human experts author skills: by holistically analyzing broad execution experience before distilling it into a single, comprehensive guide. Instead of reacting sequentially to individual trajectories, Trace2Skill dispatches a parallel fleet of sub-agents to analyze a diverse pool of executions. It extracts trajectory-specific lessons and hierarchically consolidates them into a unified, conflict-free skill directory via inductive reasoning. Trace2Skill supports both deepening existing human-written skills and creating new ones from scratch. Experiments in challenging domains, such as spreadsheet, VisionQA and math reasoning, show that Trace2Skill significantly improves upon strong baselines, including Anthropic's official xlsx skills. Crucially, this trajectory-grounded evolution does not merely memorize task instances or model-specific quirks: evolved skills transfer across LLM scales and generalize to OOD settings. For example, skills evolved by Qwen3.5-35B on its own trajectories improved a Qwen3.5-122B agent by up to 57.65 absolute percentage points on WikiTableQuestions. Ultimately, our results demonstrate that complex agent experience can be packaged into highly transferable, declarative skills -- requiring no parameter updates, no external retrieval modules, and utilizing open-source models as small as 35B parameters.

• The paper introduces a framework that extracts and consolidates trajectory-local lessons into a unified skill set using parallel, hierarchical merging.
• It demonstrates significant performance gains and cross-model transferability, achieving improvements of up to +57.7 absolute points in OOD benchmarks.
• The approach decouples skill evolution from agent parameter updates, enabling scalable and efficient development of robust procedural skills.

Trace2Skill: Distilling Trajectory-Local Lessons Into Transferable Agent Skills

Problem Setting and Motivation

The scalability and adaptability of LLM-driven agents hinge fundamentally upon the integration of domain-specific, procedural skills. Hand-authoring such skills is non-scalable and often offers brittle benefits across models and domains, as human-crafted skills might not generalize nor yield matching gains across agent architectures and task distributions. Automated skill induction from parametric knowledge is typically shallow and lacks the contextual specificity needed for robust performance in complex environments, as demonstrated by the negligible uplift of parametric skill drafts over no-skills baselines. Existing techniques for automatic skill evolution often rely on sequentially overfitting to episodic mistakes in online regimes, resulting in fragmentation and limited transferability.

Trace2Skill addresses this foundational bottleneck by introducing a formalism and algorithmic pipeline for holistic skill construction: parallelizing the extraction of trajectory-local lessons and consolidating them via inductive, conflict-aware merging. The core hypothesis is that robust, generalizable skills can be mined from sufficiently diverse agent execution experiences by treating trajectory analysis as an inductive process, yielding skills that transfer across model scales and out-of-distribution (OOD) tasks without agent parameter updates.

Methodology

Trace2Skill formalizes skill evolution as an artifact-level adaptation problem: given an agent $\pi_\theta$, a fixed initial skill $\mathcal{S}_0$ (draft or human-authored), and an evolving set of task trajectories, the objective is to construct an improved skill $\mathcal{S}^*$ such that zero-shot transfer to $\mathcal{D}_{test}$ yields measurable gains. The framework decomposes the process into three key stages:

• Trajectory Generation: The base agent operates over tasks, generating a labeled trajectory corpus partitioned into successes and failures.
• Parallel Patch Proposal (Analyst Swarm): A swarm of sub-agents processes each trajectory in isolation, producing targeted patches. Error analysts leverage a ReAct-style multi-turn workflow to causally diagnose and validate failures, while success analysts decompose effective behaviors into generalizable patterns.
• Hierarchical Conflict-Free Consolidation: All proposed patches are merged in a logarithmic hierarchy, leveraging the same LLM to perform prevalence-aware inductive reasoning and programmatic conflict elimination, resulting in a single, well-structured skill directory.

Crucially, this pipeline enables two modes: (1) skill deepening, refining a human-written prior; and (2) skill creation, evolving functional skills de novo from parametric drafts. All phases are fully parallel over tasks, and the entire system is architecturally independent of model parameter updates or external retrieval modules.

Empirical Results

In-Distribution and Cross-Model Generalization

Trace2Skill demonstrates substantial, robust gains over strong baselines on SpreadsheetBench and WikiTableQuestions (OOD). Across all model scales (Qwen3.5-35B, Qwen3.5-122B), skills distilled by Trace2Skill on one agent generalize effectively to others:

• Skill Deepening: On top of Anthropic’s human-written xlsx skill, deepening via error analysis yields up to $+$27.0 absolute points (pp) improvement for 35B agents and $+$21.5pp for 122B models, with similar cross-model transfer.
• Skill Creation from Scratch: De novo skills evolved from purely parametric drafts yield $+$22.8pp (122B) and $+$57.7pp (35B-to-122B, OOD) over baseline, matching or surpassing human-authored reference skills.

The skills exhibit strong transfer to OOD domains and unseen agents, directly contradicting prevalent assumptions that trajectory-derived experience is inherently episodic and model-bound. By contrast, episodic retrieval-based techniques (e.g., ReasoningBank (Ouyang et al., 29 Sep 2025)) show far less cross-task and cross-model generalization.

Efficiency and Structural Analysis

The parallel, many-to-one consolidation operator outperforms both sequential online updates and retrieval-memories in efficiency and performance. Hierarchical merging prevents order-dependent drift and overfitting, instead prioritizing patterns recurring across independent sub-trajectories. For instance, merge-driven Patch Consolidation completes in $\sim$3 minutes, a $20\times$ speed-up over online sequential editing, while yielding higher absolute gains.

The superior performance of the framework also derives from the agentic error analysis sub-agents, which—unlike single-call LLM error analysis—iteratively validate, debug, and patch failure traces via artifact inspection and fix verification, anchoring the resulting skills in verified causal mechanisms rather than superficial logging artifacts.

Domain Generalization: Math Reasoning and Multimodal VQA

Trace2Skill is domain-agnostic, as demonstrated by competitive uplifts in both mathematical reasoning (DAPO-Math-Test, AIME 2026) and Visual QA (DocVQA). In math, error-analyst-evolved skills yield up to $+$5pp gains across models, maintaining transferability. In multimodal VQA, skills authored by higher-capacity models provide large accuracy gains for both the native and smaller agents, pinpointing a dissociation between task execution and effective skill induction.

Analysis and Ablations

Consolidation Strategy: Parallel, hierarchical consolidation is strictly superior to sequential or batched online editing in both accuracy and engineering efficiency due to its global prevalence bias and conflict resolution mechanisms.

Retrieval Baseline Comparison: Trace2Skill's declarative skills outperform retrieval-memories, especially for small models, as retrieval degrades with test-query divergence and incurs runtime competition for LLM context/memory.

Agentic vs. Single-Call Patching: Agentic, multistep error analysis produces more causal, generalizable skill sections; single-turn LLM approaches misattribute and hallucinate error causes, reducing transferability, especially in OOD and cross-agent contexts.

Skill Structure: The nature of the induced skill directory mirrors established expert practices: high-frequency error/failure patterns emerge as explicit “critical warning” checklist items, while low-frequency or contextual quirks are modularized into references/.

Theoretical Implications and Practical Consequences

The Trace2Skill paradigm provides strong evidence that nonparametric experience can be systematically compressed into declarative, modular, transferable skills with no agent parameter updates. This enables decoupling agent model advances from skill evolution pipelines: mid-scale, open-source LLMs are sufficient to induce strong procedural skills, which can then be ported to frontier models (or vice versa).

The hierarchical, many-to-one inductive strategy can be interpreted as in-silico analog to human expert codification of procedural knowledge, effectively operationalizing large-scale domain-generalization via high-throughput, conflict-aware abstract pattern mining. This is of significant importance as the agent-skills ecosystem expands, favoring approaches that maximize transfer, minimize fragmentation, and permit human inspection and pruning.

Limitations and Future Directions

The current pipeline applies patch consolidation holistically, and as such, marginal contributions or interferences between specific skill edits are not causally isolated, impeding precise attribution and automated pruning. Trace utility tracing over the agent’s inference process (tracking which skill sections are actually invoked and beneficial) is also open for future research. These advancements would enable more technical rigor in skill refinement, e.g., counterfactual editing and section-wise attribution, and tighter integration with agent harnesses.

Conclusion

Trace2Skill demonstrates that systematic, large-scale mining of agent execution trajectories—processed in parallel and hierarchically merged—yields artifact-level skills that are demonstrably transferable across model architectures, agent scales, and task distributions. The pipeline achieves these gains without reliance on proprietary LLMs, without fine-tuning, and without runtime retrieval or memory modules. This work provides a clear blueprint for scalable skill evolution and codification, relevant for practitioners seeking robust, portable foundation agent skills (2603.25158).