WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution

Abstract: Agent skills package specialized knowledge and workflows into reusable resources that extend AI agent capabilities. Recent work automatically discovers such skills from agent experience, which enables agents to progressively adapt through interaction. However, the insights that guide skill development typically remain scattered across optimization histories, limiting their systematic reuse across iterations. We introduce WikiSkill, a framework that co-evolves agent skills with a persistent knowledge base (wiki). At a high level, WikiSkill separates raw execution experience, accumulated knowledge, and executable skills, while continuously consolidating experience into the wiki, which subsequent skill updates can build on. Across diverse benchmarks and models, WikiSkill consistently outperforms state-of-the-art skill-evolution methods and improves over no-skill baselines in most model-benchmark settings. We find that skill evolution complements model scaling: larger models generally benefit more from evolved skills, while smaller models with skills can outperform substantially larger models without them. We also find that evolved skills transfer effectively across models and model families, and skills evolved by other models can outperform self-evolved skills. Finally, our ablation studies confirm that persistent knowledge accumulation in the wiki is critical for effective skill evolution. These results demonstrate the benefits of systematically accumulating and refining agent experience for developing reusable and transferable skills.

• * The paper WikiSkill introduces a novel agent skill evolution method, boosting performance by separating knowledge accumulation from skill execution, outperforming existing methods across various domains, reveals that stronger models benefit more from procedural knowledge*
• * WikiSkill separates execution traces from procedural skills via a wiki layer, maintaining structured, persistent diagnostic knowledge that encourages systematic improvements
• WikiSkill demonstrates that inferring without the structured knowledge worsens performance, highlighting the wiki’s critical role in guiding skill evolution

WikiSkill addresses a specific weakness in experience-driven agent adaptation: execution traces and optimization histories may contain useful information, but prior skill-evolution systems generally do not maintain that information as an independent, structured, persistent knowledge representation. The paper introduces a three-layer architecture in which raw experience is preserved, observations are consolidated into a persistent wiki, and only validated procedural knowledge is exposed to the inference agent as executable skills (2608.27454). The central claim is that skill evolution improves when the system separates knowledge accumulation from procedural deployment.

Research problem and central contribution

Agent skills are filesystem-based modules containing instructions, applicability conditions, scripts, and auxiliary resources. They provide a parameter-free mechanism for encoding domain-specific procedures and can be reused across tasks and models. The paper focuses on automatically evolving these skills from interaction data rather than manually authoring them.

Existing methods such as EvoSkill, Trace2Skill, and SkillOpt already follow an iterative pattern: execute an agent, inspect successes and failures, propose a modification to a skill, and retain the modification if validation performance improves. WikiSkill argues that this pipeline conflates at least two different functions. The first is discovering and organizing procedural knowledge from experience; the second is executing that knowledge effectively during inference. In prior systems, information remains distributed across rejected proposals, trace analyses, optimizer state, and historical feedback. Consequently, later proposals may lack a coherent account of recurring failures, successful strategies, or previously rejected interventions.

WikiSkill introduces a persistent wiki as an intermediate representation between raw trajectories and active skills. This design is the paper’s principal methodological contribution. The wiki is not itself injected into the inference agent during task execution. Instead, it supports the maintainers and proposers responsible for evolving skills.

Figure 1: WikiSkill separates immutable execution traces, persistent structured knowledge, and active procedural skills within an iterative evolution loop.

Three-layer architecture and evolution loop

The workspace is divided into three layers. The Raw Layer stores immutable execution trajectories, including observations, actions, tool calls, tool outputs, reasoning traces, and final answers. Immutability preserves an audit trail and permits later reanalysis without losing the original evidence.

The Wiki Layer contains structured pattern pages, an index, an evolution log, and a skill-impact tracker. Pattern pages describe recurring failure modes or successful strategies, including root-cause analyses and concrete workarounds. The evolution log records iteration-level findings, while the skill-impact tracker records proposal diffs, validation scores, and acceptance or rejection decisions. Importantly, the wiki is retained even when a candidate skill is rolled back.

The Skills Layer contains the currently active procedural modules. Each skill has a SKILL.md file with instructions and applicability conditions, together with a PURPOSE.md file linking the skill to the wiki patterns that motivated its creation or modification. The inference agent receives the complete active skill content through prompt injection. This experimental choice deliberately removes skill retrieval and triggering from the evaluation, isolating the quality of the evolved procedures.

Each iteration consists of four stages. First, the Inference Agent executes the training tasks using the current skills but cannot access the wiki. Second, the Wiki Maintainer samples successful and failing traces, performs root-cause analysis, and creates or incrementally edits pattern pages. Third, the Skill Proposer operates in ReAct mode, reads the wiki index and historical impact records, selectively inspects relevant patterns and traces, and produces an atomic proposal affecting one skill. Finally, the candidate skill is evaluated on validation tasks. It is accepted only if its validation score strictly exceeds the best score observed so far; otherwise, the skill set is rolled back while the wiki remains unchanged.

The strict gating rule provides a simple protection against immediate regressions, but it also imposes a myopic criterion: a neutral proposal that preserves performance but enables later improvements is rejected. The paper explicitly adopts this rule for comparability with prior systems rather than claiming it is optimal.

Experimental design

The evaluation spans five task domains:

• LiveMathematicianBench for mathematical reasoning;
• SealQA for search-augmented factual question answering;
• SpreadsheetBench for executable spreadsheet manipulation;
• OfficeQA for long-context document retrieval and grounded reasoning;
• ALFWorld for interactive embodied task completion.

Five inference models are evaluated: Qwen-3.5-4B, Qwen-3.5-9B, Qwen-3.6-27B, Gemma-4-31B, and Gemini-3.5-Flash. WikiSkill is compared with Trace2Skill, EvoSkill, SkillOpt, and a no-skill baseline. Every evolution process is repeated three times, with reported test results averaged across runs. The paper uses paired bootstrap testing with 1,000 iterations and a significance threshold of $p < 0.05$.

The training splits are small: 16–80 examples depending on the benchmark, with validation sets ranging from 10 to 40 examples. The authors acknowledge that this makes validation gating noisy. Repeated full-pipeline runs partially address the issue, but they do not eliminate the possibility that acceptance decisions overfit small validation sets.

Main performance results

WikiSkill obtains the highest macro-average test performance for every evaluated model. The results are summarized below.

Inference model	No skill	Best competing method	WikiSkill	WikiSkill gain over no skill
Qwen-3.5-4B	26.2	35.2	38.5	+12.3
Qwen-3.5-9B	29.9	42.3	47.4	+17.5
Qwen-3.6-27B	39.4	53.3	63.3	+23.9
Gemma-4-31B	41.3	49.1	54.9	+13.6
Gemini-3.5-Flash	49.5	56.1	68.1	+18.6

WikiSkill’s advantage over the strongest competing evolution method is 3.3, 5.1, 10.0, 5.8, and 12.0 percentage points for the five models, respectively. These margins are not uniform across domains, but the method is more consistent than the baselines. For example, EvoSkill raises Qwen-3.5-9B’s LiveMath performance from 28.2% to 58.1%, yet reduces Gemma-4-31B’s performance on the same benchmark from 33.9% to 29.8%. SkillOpt similarly reduces Gemini-3.5-Flash’s SealQA score from 29.4% to 28.2%. WikiSkill avoids these particular regressions while also achieving larger average gains.

Figure 2: WikiSkill’s average accuracy exceeds the no-skill baseline and competing skill-evolution methods, with larger advantages for stronger models.

Several individual improvements are especially large. Gemini-3.5-Flash increases from 33.0% to 72.6% on LiveMath and from 50.5% to 76.6% on SpreadsheetBench. Qwen-3.6-27B improves from 52.8% to 77.6% on ALFWorld and from 40.8% to 81.7% on SpreadsheetBench. These results imply that procedural skill evolution is useful not only for interactive environments but also for single-step reasoning and tool-mediated code execution.

The pattern across datasets is heterogeneous. For Qwen-3.6-27B, WikiSkill improves performance by 28.0 points on LiveMath, 14.1 points on SealQA, 40.9 points on SpreadsheetBench, 11.6 points on OfficeQA, and 24.8 points on ALFWorld. OfficeQA is comparatively resistant for smaller models: Qwen-3.5-4B declines slightly from 30.2% to 28.5%. The paper attributes this to the model’s difficulty executing detailed multi-step retrieval procedures over long contexts. Thus, the result is not that skills universally compensate for weak inference capability; rather, the benefit depends on whether the model can operationalize the procedure encoded in the skill.

Interaction with model scaling

A prominent claim is that skill evolution complements, rather than substitutes for, model scaling. Within the Qwen family, WikiSkill’s average gains increase from 12.3 points for Qwen-3.5-4B to 17.5 points for Qwen-3.5-9B and 23.9 points for Qwen-3.6-27B. SpreadsheetBench exhibits the clearest scaling interaction: gains are 6.5, 9.3, and 40.9 points for the 4B, 9B, and 27B models.

The implication is that stronger models appear better able to interpret, adapt, and execute complex procedural instructions discovered during evolution. However, the paper also presents a substantial scale-compensation effect. Qwen-3.5-9B with WikiSkill reaches 47.4% average accuracy, exceeding Qwen-3.6-27B without skills at 39.4%. Qwen-3.5-4B with WikiSkill reaches 38.5%, nearly matching the unskilled 27B model.

This interaction complicates a simple interpretation of model scaling. Parameter count remains strongly relevant, but reusable procedural knowledge changes the effective capability of a model on task distributions with stable workflows. The evidence supports a decomposition in which model capability determines both skill discovery and skill execution, while the skill provides an externalized procedural prior.

Cross-model transfer and negative transfer

WikiSkill also evaluates whether a skill evolved by one model can be used by another. The results show substantial transfer across model scales and families, including cases where transferred skills outperform self-evolved skills.

For example, a skill evolved by Qwen-3.6-27B raises Qwen-3.5-9B’s SpreadsheetBench score from 24.3% without skills to 50.5%, compared with 33.6% using the smaller model’s self-evolved skill. The same source skill raises Gemma-4-31B’s LiveMath score from 33.9% to 73.7%, above its self-evolved result of 56.7%. Transfer also occurs in the opposite direction: Qwen-3.5-4B skills raise Gemma-4-31B’s LiveMath performance to 73.1% and ALFWorld performance to 66.9%.

These results support the paper’s distinction between skill discovery and skill execution. A model may discover a useful procedure but be unable to execute it reliably itself. Conversely, a stronger target model may exploit a procedure that the source model discovered under its own execution constraints.

The transfer results also reveal a major qualification: skills can produce severe negative transfer when they encode model-specific workarounds rather than general procedures. Qwen-3.5-4B’s SpreadsheetBench skill reduces Gemini-3.5-Flash from 50.5% to 18.1%, whereas Qwen-3.6-27B’s skill raises it to 63.4%. The reported explanation is that the smaller model’s skill contains low-level execution constraints, including single-line Python commands and string-conversion rules, that prevent a stronger model from using more effective end-to-end scripts. Fragmented diagnostic routines can also generate redundant tool calls and exhaust the target model’s interaction budget.

This finding limits the interpretation of transferability. A skill is not necessarily a model-independent abstraction merely because it is represented as text. Its contents may reflect the source model’s failure modes, action budget, tool-use tendencies, or reasoning limitations.

Figure 3: An ALFWorld case study shows how rejected proposals, accepted edits, and later failure evidence accumulate in the wiki to guide successive skill refinements.

Persistent knowledge as the source of improvement

The ablation study isolates the contribution of wiki persistence and wiki access. Using Gemini-3.5-Flash, the default configuration gives the Skill Proposer access to the wiki while withholding it from the Inference Agent. This configuration achieves a 63.7% average across the evaluated benchmarks.

When both the Inference Agent and Skill Proposer lack wiki access, the average falls to 48.7%. Giving the Proposer access to the persistent wiki therefore produces a 15.0-point improvement. The largest gains occur on LiveMath, which rises from 51.3% to 72.6%, and SpreadsheetBench, which rises from 49.9% to 76.6%.

The result directly supports the paper’s central hypothesis: the benefit does not come merely from adding another textual context to the system. It comes from retaining structured evidence across iterations, including recurring errors, successful strategies, rejected proposals, and objective validation outcomes. The wiki allows later proposals to avoid repeating failed interventions and to connect new observations with previously documented patterns.

The ablation also produces a counterintuitive result. Giving the Inference Agent access to the wiki during training rollouts reduces average performance from 63.7% to 60.9% when the Proposer retains wiki access. LiveMath falls from 72.6% to 64.8%. The paper hypothesizes that the agent may solve tasks by consulting wiki knowledge directly rather than by using the active skills, making the resulting trajectories less diagnostic of skill quality. Under this interpretation, more information during training can reduce the quality of the experience used for procedural compilation.

This conclusion depends on the specific access configuration and prompt design. It does not establish that wiki access is harmful for inference generally; it establishes that allowing the training-time Inference Agent to bypass the active skill layer can weaken the signal used for skill evolution.

Knowledge accumulation and qualitative dynamics

WikiSkill produces relatively concise executable skills while retaining more expansive diagnostic knowledge in the wiki. Across models, Qwen skills average approximately 119–129 Markdown lines, compared with 45 lines for Gemma-4-31B and 81 lines for Gemini-3.5-Flash. Across benchmarks, SpreadsheetBench generates the longest skills at 142.5 lines and the largest number of wiki patterns, averaging 9.8 created patterns. LiveMath generates shorter skills averaging 84.6 lines and fewer patterns, averaging 4.4.

Accepted updates continue beyond the first iterations. Depending on the model, 39–52% of accepted updates occur during iterations 0–1, with additional accepted changes in both middle and late stages. SealQA is particularly dependent on prolonged refinement: 33% of accepted updates occur in iterations 2–4 and 28% in iterations 5–7. This temporal distribution is consistent with the proposed role of persistent knowledge: later changes are informed not only by fresh traces but also by accumulated evidence and prior acceptance decisions.

The ALFWorld case study makes this mechanism concrete. An initial proposal called goal-directed-action is rejected. The rejection is preserved in skill-impact.md. The next iteration produces break-repetition-loop, including the rule “Never Return an Item to Its Origin Location,” which is accepted. Later traces reveal a broader class of multi-operation loops, leading to a refinement requiring each operation type to be applied only once per item. The case demonstrates that the wiki is not simply a memory of successful behavior; it records failed hypotheses and uses them as constraints on subsequent proposal generation.

Computational cost and systems assumptions

The paper argues that WikiSkill has favorable optimizer-call scaling under its experimental full-batch configuration. The Wiki Maintainer makes one call per batch, and the ReAct Skill Proposer uses approximately 10–20 calls per iteration. With batch size equal to the full training set, the optimizer call count is independent of the number of training examples.

This comparison should be interpreted carefully. WikiSkill’s apparent $\mathcal{O}(1)$ dependence on training-set size arises because the experiments use full-batch processing and because the Proposer selectively reads traces on demand. The inference rollouts themselves still scale with the number and length of tasks, and the ReAct interaction introduces a nontrivial fixed cost. Moreover, the comparison favors WikiSkill’s chosen full-batch setting against baselines whose strongest configurations use minibatches. The relevant systems trade-off is therefore not simply call complexity but the quality, latency, and token cost of centralized trace inspection versus distributed analysis.

Limitations and open questions

The evaluation directly injects all active skills into the inference prompt and does not study skill retrieval, triggering, ranking, or interference among multiple skills. As a result, the reported gains concern skill quality under oracle provisioning rather than complete skill-library operation.

The strict validation rule accepts only proposals that produce immediate score improvements. This excludes neutral updates that might enable later improvements and may favor short-term conservative edits. The small validation splits further introduce noise into these decisions, although the authors average over three independent evolution runs and apply paired bootstrap testing.

The wiki grows monotonically. Pattern pages, logs, and proposal diffs are retained indefinitely, but the framework has no pruning, deduplication, contradiction resolution, or relevance-based compression mechanism. Long evolution runs could therefore produce retrieval overhead, stale knowledge, or conflicts between patterns. The paper also evaluates only relatively short-horizon tasks. It does not test online skill refinement during executions lasting hundreds of actions or several hours.

Several questions remain specific to the framework. How should wiki evidence be compressed without deleting rare but high-value failure modes? Can gating criteria accommodate delayed benefits while controlling validation overfitting? Under what conditions can a proposer distinguish model-general procedures from source-model-specific workarounds before negative transfer occurs? And how would the architecture behave when a large skill library requires retrieval rather than full prompt injection?

Conclusion

WikiSkill’s contribution is an explicit separation between immutable experience, persistent diagnostic knowledge, and executable procedural skills. Across five benchmarks and five models, it improves average performance over both no-skill systems and established skill-evolution baselines, with gains increasing from 12.3 to 23.9 points across Qwen model scales. The strongest evidence for the proposed mechanism comes from the ablation: persistent wiki access for the Skill Proposer improves Gemini-3.5-Flash’s average score by 15.0 points, while granting the wiki directly to the training-time Inference Agent reduces performance.

The results establish that persistent experience compilation can materially improve parameter-free skill evolution. They also show that transfer is neither automatically beneficial nor determined solely by source-model strength: procedural abstractions transfer well when they capture general workflows, but model-specific workarounds can cause severe negative transfer. WikiSkill therefore provides both an effective evolution pipeline and a clear empirical separation between discovering procedures and executing them.