WikiSkill Framework

• The WikiSkill framework compiles agent execution experience into persistent, structured knowledge for reusable and evolvable skills, enhancing agent capability development through structured traces to skill updates.
• It separates raw execution trajectories, monitored patterns, and executable skills using a wikibased knowledge layer, accommodates diverse domains, and supports systematic skill evolution in benchmarks across mathematical reasoning, web search, and procedural tasks.
• WikiSkill outperforms other skill-evolution models in benchmark evaluations, maintaining the highest average performance across metrics, especially benefiting from the cross-model transfer, proving the framework transfers effectively.

WikiSkill is a framework for compiling agent execution experience into persistent, structured knowledge that supports the evolution of reusable executable skills. Introduced in “WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution” (Tang et al., 27 Aug 2026), it separates raw execution trajectories, accumulated explanatory knowledge, and active filesystem-based skills. A Wiki Maintainer consolidates successes, failures, root causes, and workarounds into a persistent wiki; a Skill Proposer uses that knowledge and recent traces to create or patch skills; and validation gating accepts only changes that strictly improve validation performance. Unlike methods that retain knowledge primarily in proposal histories or directly modify skill documents, WikiSkill preserves rejected proposals and recurring failures in a monotonic knowledge layer.

1. Conceptual foundations and scope

WikiSkill addresses the problem that agents can acquire specialized procedural knowledge during interaction, but the insights guiding subsequent skill updates often remain scattered across optimization histories, trace analyses, rejected edits, or transient reasoning. Modern agents require domain-specific procedures including search strategies, tool workflows, spreadsheet manipulation conventions, mathematical heuristics, document-navigation procedures, and embodied action policies. Filesystem-based skills provide a lightweight alternative to parameter updates by packaging instructions, scripts, metadata, and resources into auditable directories.

The framework treats skill evolution as an infrastructure and knowledge-management problem rather than solely as a skill-generation problem. Its central design separates:

• Raw execution experience: immutable step-by-step trajectories containing observations, actions, tool calls, tool outputs, reasoning, and final answers.
• Persistent wiki knowledge: structured and retrievable patterns derived from experience, including explanations of failures, root causes, successful strategies, and workarounds.
• Executable skills: active procedural modules injected into the inference agent at execution time.

The resulting information flow is:

$$\text{raw traces} \rightarrow \text{wiki patterns and history} \rightarrow \text{candidate skill updates} \rightarrow \text{validation} \rightarrow \text{accepted skill or rollback}.$$

The framework is related to broader infrastructures for collaborative knowledge, governed artifacts, and agent capability management. “Collaborative Structuring of Knowledge by Experts and the Public” (Morris et al., 2010) describes Citizendium as a system separating open contribution from expert review, stable approved content from evolving drafts, and knowledge production from governance. “SkillWiki: A Living Knowledge Infrastructure for Agent Skills” (Huang et al., 15 Jun 2026) develops a related infrastructure perspective in which agent skills are versioned, provenance-aware, governed, and connected to execution experience. WikiSkill differs from these systems by specifying and evaluating a concrete iterative mechanism for compiling traces into persistent knowledge and executable skill updates.

WikiSkill is not a differentiable optimizer. Its optimization process is LLM-mediated proposal search combined with black-box validation selection. The practical objective is to evolve the joint state of active skills and persistent knowledge so as to improve performance on unseen tasks.

2. Layered architecture and data model

The framework represents its state at iteration $k$ as $(S_k,W_k)$, where $S_k$ is the active executable skill set and $W_k$ is the persistent wiki. Initially,

$(S_0,W_0)=(\varnothing,\varnothing).$

A task dataset is partitioned into training, validation, and test splits:

$$D = D_{\mathrm{train}} \cup D_{\mathrm{val}} \cup D_{\mathrm{test}}.$$

For task $x_i$, an agent $\pi$ produces a trajectory

$\tau_i \sim \pi(x_i;S),$

where the trajectory contains alternating observations and actions, and the final action emits a prediction $k$0. If $k$1 is the task-scoring function, performance on split $k$2 is:

$k$3

Raw layer

The raw layer is stored under raw/. It preserves complete execution traces rather than only summaries or extracted skill instructions. These traces provide empirical evidence for root-cause analysis, including:

• observations and actions;
• tool calls and tool outputs;
• commands and feedback;
• reasoning and final answers;
• successful and failed execution sequences;
• recovery attempts and reflection memories.

Raw traces are immutable in the framework’s conceptual design. This preserves the evidence from which later wiki updates and skill proposals can be audited.

Wiki layer

The persistent wiki is stored under wiki/ and contains:

• wiki/index.md, a concise catalog of available patterns;
• wiki/patterns/, containing one Markdown page per meaningful success or failure pattern;
• wiki/log.md or wiki/logs.md, containing chronological evolution history and iteration summaries;
• wiki/skill-impact.md, recording proposals, diffs, validation scores, and Accepted or Rejected outcomes.

The supplied paper uses both wiki/log.md and wiki/logs.md in different descriptions. The maintainer prompt specifies wiki/log.md, whereas the architecture description specifies logs.md.

Pattern pages are intended to contain descriptions, root-cause analyses, exact command or action sequences, and concrete solutions or workarounds. The Wiki Maintainer documents both failures and successes and avoids treating the wiki as a simple error log. It is an explanatory layer between experience and executable behavior.

Skill layer

Active skills are stored under skills/. Each skill contains:

• SKILL.md, with YAML frontmatter, applicability conditions, procedural instructions, and potentially scripts or other resources;
• PURPOSE.md, linking the skill to the wiki patterns that motivated it and recording its origin, addressed patterns, and evolution history.

Skills are the only layer directly injected into the inference agent in the main experiments. The inference agent does not directly access the wiki during execution in the default configuration. This separation prevents the agent from solving tasks using uncompiled wiki information and thereby producing traces that are less informative for skill development.

A skill is conceptually a lifecycle-aware capability asset with identity, classification, state, interface, implementation, evaluation contract, provenance, relations, and runtime information. WikiSkill does not provide a complete formal JSON schema or database schema, but its abstraction can be represented as:

$k$4

This notation is an explanatory abstraction rather than a formal schema reported by the paper.

3. Evolution algorithm and governance

WikiSkill uses a recurrent pipeline consisting of inference, trace sampling, wiki maintenance, skill proposal, validation, and audit updating.

Initialization

The system begins with no active skills and no wiki. It evaluates the empty-skill agent on the validation split and stores the resulting score as $k$5.

Inference

At iteration $k$6, the inference agent executes the training tasks using the currently accepted skills:

$k$7

The full training trajectories are written to the raw layer. The default inference agent receives the full contents of active skills through its system prompt but cannot access the wiki.

Trace sampling

To control context-window pressure, the system selects up to eight training traces per iteration:

• at most five failures;
• at most three successes.

Each injected execution log is capped at 15,000 characters. Failure traces provide diagnostic evidence, while successful traces preserve strategies that may prevent regressions.

Wiki maintenance

The Wiki Maintainer reads the prior wiki and sampled traces. It performs root-cause analysis, extracts generalizable action patterns, creates or patches pattern pages, rewrites the complete index, and appends an iteration summary. The transition is:

$k$8

Pattern pages can be modified incrementally using append, replace, and insert_after patch operations. The maintainer is required to inspect actual actions, compare successes and failures, identify generalizable patterns, and determine whether current skill guidance helped.

Skill proposal

The Skill Proposer is a multi-turn ReAct agent. It first reads wiki/index.md and wiki/skill-impact.md, then retrieves relevant pattern pages and raw traces with read_file, and finally submits one atomic proposal. It must read at least four execution traces before proposing a change and should prefer patching a partially correct skill.

The proposal is represented abstractly as:

$k$9

There are three proposal types:

• Create: create a new skill directory containing SKILL.md and PURPOSE.md.
• Patch: modify an existing skill using append, replace, or insert_after operations.
• No action: make no skill change.

Validation gating

The proposal is applied to the active skill set:

$(S_k,W_k)$0

The candidate is evaluated on the validation split. It is accepted only if validation performance strictly improves:

$(S_k,W_k)$1

If the condition holds, $(S_k,W_k)$2 and $(S_k,W_k)$3 is updated. Otherwise, $(S_k,W_k)$4 and the candidate skill change is rejected. Evolution stops if $(S_k,W_k)$5.

Only the skill set is rolled back. The wiki remains updated, including evidence associated with rejected proposals. The audit transition is:

$(S_k,W_k)$6

where $(S_k,W_k)$7 is Accepted or Rejected.

This asymmetric rollback is central to WikiSkill. Executable behavior is performance-gated and reversible, whereas explanatory knowledge and historical evidence accumulate across iterations.

4. Skill taxonomy, provenance, and runtime evolution

WikiSkill organizes skills into three hierarchical categories:

1. Atomic skills: basic operations.
2. Functional skills: reusable task-level capabilities.
3. Strategic skills: higher-level planning, coordination, or orchestration capabilities.

The taxonomy supports search, composition, and large-scale organization, although the experiments do not evaluate skill retrieval or triggering. Active skills are fully injected into the inference prompt.

The framework’s persistent knowledge is intended to preserve reusable patterns rather than merely retain raw trajectories. A pattern may describe a recurring failure, a successful operational sequence, a tool-use convention, or a domain-specific workaround. The Skill Proposer can use these patterns to construct minimal, auditable modifications to active skills.

Runtime health

Skill execution produces runtime evidence, including execution counts, success rates, latency, failure patterns, recovery attempts, and reflection memories. For a skill $(S_k,W_k)$8, the intended success rate is:

$(S_k,W_k)$9

and the failure rate is:

$S_k$0

The supplied LaTeX contains formatting errors, but these expressions represent the intended definitions. The paper does not specify a complete health-scoring function or degradation threshold.

Repeated failures and recovery attempts generate reflection memories. These memories become evidence for proposed refinement, repair, decomposition, composition, replacement, deprecation, or retirement.

The lifecycle states are:

• Raw Experience
• Candidate
• Draft
• Verified
• Released
• Degraded
• Deprecated
• Archived

The demonstrated lifecycle includes production, verification, release, execution, repair, versioning, deprecation, and archival. This lifecycle distinguishes the currently executable artifact from historical and rejected alternatives.

The architecture is related to the provenance and governance model described in “SkillWiki: A Living Knowledge Infrastructure for Agent Skills” (Huang et al., 15 Jun 2026). That system treats skills as governed assets connected to knowledge sources, tools, executions, validations, versions, dependencies, maintenance proposals, and evolution events. WikiSkill provides a more specific experimental formulation of the knowledge-to-skill loop, particularly the separation between persistent explanatory knowledge and performance-gated executable skills.

5. Computational characteristics and benchmark methodology

WikiSkill uses full-batch training in the reported experiments. Let $S_k$1 denote optimizer LLM API calls per evolution iteration, $S_k$2 the training-set size, $S_k$3 the minibatch size, and $S_k$4 the number of proposer ReAct turns. Its optimizer-call complexity is:

$S_k$5

With full-batch training, $S_k$6, giving:

$S_k$7

The paper reports approximately $S_k$8. Thus, optimizer-call growth is $S_k$9 with respect to training-set size under full-batch operation, although each call can involve substantial context reading and reasoning.

For comparison, the reported complexity forms are:

• EvoSkill: $W_k$0;
• SkillOpt: $W_k$1, with $W_k$2–$W_k$3;
• Trace2Skill: approximately $W_k$4, where $W_k$5 is the reduction-tree branching factor.

WikiSkill therefore uses selective trace retrieval and a full-batch proposer rather than one optimizer call per training trajectory. The paper notes that this can reduce optimizer-call growth while imposing a potentially higher per-call context cost.

Benchmarks

The experiments cover five benchmarks:

Benchmark	Domain	Test size
LiveMath	Mathematical reasoning	124
SealQA	Web search and factual question answering	85
SpreadSheet	Spreadsheet manipulation	280
OfficeQA	Long-context document question answering	172
ALFWorld	Interactive embodied household tasks	134

The training and validation sizes are:

• LiveMath: 35 train, 18 validation;
• SealQA: 16 train, 10 validation;
• SpreadSheet: 80 train, 40 validation;
• OfficeQA: 50 train, 24 validation;
• ALFWorld: 39 train, 18 validation.

The models are:

• Qwen-3.5-4B-Instruct;
• Qwen-3.5-9B-Instruct;
• Qwen-3.6-27B;
• Gemma-4-31B;
• Gemini-3.5-Flash.

Open-weight Qwen and Gemma models were deployed with vLLM, while Gemini was used as a closed model. Baselines were no skill, Trace2Skill, EvoSkill, and SkillOpt. All methods began with an empty skill set, injected evolved skills into the inference prompt at test time, and used a fixed broader agent harness. Each full evolution process was run three times. Scores are averages over the three evolved skill sets, and paired bootstrap tests used 1,000 iterations.

Main results

The reported metric is test accuracy or benchmark score in percent, with the average computed across the five benchmarks.

Model	No skill average	Best competing evolution average	WikiSkill average
Qwen-3.5-4B	26.2	35.2	38.5
Qwen-3.5-9B	29.9	42.3	47.4
Qwen-3.6-27B	39.4	53.3	63.3
Gemma-4-31B	41.3	49.1	54.9
Gemini-3.5-Flash	49.5	56.1	68.1

WikiSkill has the highest reported average score in every model block. Its average gain over the strongest competing skill-evolution method is:

• Qwen-3.5-4B: +3.3 points;
• Qwen-3.5-9B: +5.1 points;
• Qwen-3.6-27B: +10.0 points;
• Gemma-4-31B: +5.8 points;
• Gemini-3.5-Flash: +12.0 points.

Selected improvements over no-skill baselines include:

• Gemini-3.5-Flash LiveMath: 33.0 to 72.6;
• Gemini-3.5-Flash SpreadSheet: 50.5 to 76.6;
• Qwen-3.6-27B SpreadSheet: 40.8 to 81.7;
• Qwen-3.6-27B ALFWorld: 52.8 to 77.6.

The results indicate that WikiSkill is more consistent across model–benchmark pairs than the evaluated baselines. The paper reports that some competing methods improve particular pairs but degrade others, whereas WikiSkill improves most pairs and achieves the highest average in every model block.

6. Scaling, transfer, and ablation findings

Scaling with model capability

Within the Qwen family, WikiSkill’s average gains over no skill are:

• Qwen-3.5-4B: +12.3 points;
• Qwen-3.5-9B: +17.5 points;
• Qwen-3.6-27B: +23.9 points.

The gain increases with scale, especially on SpreadSheet:

• Qwen-3.5-4B: +6.5 points;
• Qwen-3.5-9B: +9.3 points;
• Qwen-3.6-27B: +40.9 points.

The authors interpret these results as evidence that skill discovery and skill execution both benefit from model capability. The framework also permits smaller models with skills to outperform larger models without them: Qwen-3.5-9B with WikiSkill scores 47.4 average, exceeding Qwen-3.6-27B without skills at 39.4.

Skill gains are largest on structured, procedural, or tool-heavy tasks. For Qwen-3.6-27B, improvements over no skill are +28.0 on LiveMath, +14.1 on SealQA, +40.9 on SpreadSheet, +11.6 on OfficeQA, and +24.8 on ALFWorld.

Cross-model transfer

Skills evolved by one model often transfer to another model or model family, and externally evolved skills can outperform self-evolved skills. Reported examples include:

• On ALFWorld with Qwen-3.5-9B, no skill scores 34.7, the self-evolved skill scores 63.4, and a Qwen-3.6-27B skill scores 70.2.
• On LiveMath with Gemma-4-31B, no skill scores 33.9, its self-evolved skill scores 56.7, and a Qwen-3.6-27B skill scores 73.7.
• On LiveMath with Gemini-3.5-Flash, no skill scores 33.0, a Qwen-3.5-4B skill scores 67.5, a Qwen-3.6-27B skill scores 73.9, and the self-evolved skill scores 72.6.
• On SpreadSheet with Qwen-3.6-27B, no skill scores 40.8, its self-evolved skill scores 81.7, and a Gemini-3.5-Flash skill scores 76.0.

Transfer is not exclusively from stronger source models to weaker targets. Qwen-3.5-4B skills improve Gemma-4-31B to 73.1 on LiveMath and 66.9 on ALFWorld.

Negative transfer is also observed. Qwen-3.5-4B skills reduce Gemini-3.5-Flash SpreadSheet performance from 50.5 to 18.1. The paper attributes this to model-specific workarounds, including restrictive low-level instructions such as single-line Python commands and string-conversion rules, as well as fragmented diagnostic steps that consume the stronger model’s interaction budget.

These results support portability but do not imply universal model independence. Skill specifications can encode assumptions about capabilities, action policies, interaction budgets, or tool behavior.

Persistent-wiki ablation

Using Gemini-3.5-Flash, the ablation independently varies wiki access for the training-time inference agent and the Skill Proposer. When the Proposer lacks wiki access, the Wiki Maintainer is also removed.

Inference-agent wiki access	Proposer wiki access	Average excluding ALFWorld
No	No	48.7
No	Yes	63.7
Yes	No	45.3
Yes	Yes	60.9

Persistent wiki access for the Proposer is critical: with no wiki access for the inference agent, enabling the Proposer and Maintainer raises the average from 48.7 to 63.7, a +15.0-point gain. Giving the training-time inference agent direct wiki access reduces the average from 63.7 to 60.9 when Proposer access is enabled.

The authors hypothesize that direct wiki access allows the training agent to solve tasks using knowledge that is not encoded in the skill, making traces less diagnostic for skill development. The ablation consequently supports the architectural separation in which the wiki guides skill construction while the executing agent is evaluated through the active skill interface.

Qualitative ALFWorld case

The ALFWorld case study illustrates how persistent knowledge affects later proposals:

1. At iteration 0, the Wiki Maintainer identifies a take-examine-move-loop pattern.
2. The Proposer creates goal-directed-action, but validation does not improve and the proposal is rejected.
3. skill-impact.md preserves the rejected diff and outcome.
4. At iteration 1, informed by the rejection, the Proposer creates break-repetition-loop with the rule “Never Return an Item to Its Origin Location”; the update is accepted.
5. Later traces reveal a related multi-operation-loop pattern.
6. At iteration 4, the Proposer refines the skill with “Each Operation Type ONCE Per Item.”

The example demonstrates the division of labor between raw traces, persistent patterns, rejection history, and the currently selected executable rule.

7. Limitations and research significance

WikiSkill’s principal contribution is architectural and procedural: it introduces a durable explanatory layer between trajectories and skills. Raw experience supplies evidence, the Wiki Maintainer transforms evidence into indexed root-cause-oriented patterns, the Skill Proposer retrieves and reasons over those patterns, validation gating protects execution quality, and the wiki retains rejected knowledge.

Several limitations constrain interpretation.

• No skill retrieval evaluation: active skills are fully injected into the prompt, so behavior with large skill libraries, triggering mechanisms, and retrieval errors remains untested.
• Strict validation gating: neutral proposals are rejected even though they might enable later improvements.
• Validation noise: small validation sets can cause useful proposals to be rejected or harmful proposals to be accepted.
• Wiki growth: the wiki has no automated pruning mechanism, so long runs may create redundancy, stale knowledge, and increasing context costs.
• Limited horizon: the benchmarks do not cover extremely long-horizon tasks involving hundreds of actions or hours of interaction.
• Negative transfer: model-specific procedural workarounds can make externally evolved skills harmful.
• Execution complexity: overly detailed or fragmented skills can be difficult for smaller models to execute and may exhaust interaction budgets.
• Incomplete low-level specification: the paper does not provide the full skill schema, graph implementation, retrieval mechanism, permission architecture, security controls, automatic test-generation procedure, rollback implementation, or formal degradation thresholds.
• Limited empirical scope: there are no ablation studies beyond the reported wiki-access experiment, no baseline comparisons on large-scale deployments, no confidence intervals for all reported results, and no long-term stability study.

The 99/125 feasibility result reported in “SkillWiki: A Living Knowledge Infrastructure for Agent Skills” (Huang et al., 15 Jun 2026) concerns conversion of heterogeneous artifacts into governed candidates and is distinct from WikiSkill’s benchmark results. WikiSkill evaluates iterative skill evolution across LiveMath, SealQA, SpreadSheet, OfficeQA, and ALFWorld, whereas SkillWiki presents a broader lifecycle infrastructure involving ingestion, provenance-aware exploration, governance, execution feedback, repair, versioning, deprecation, and archival.

The central research implication is that reusable procedural knowledge may require an explicit persistent representation independent of executable skill documents. Skills must remain concise enough for execution, while the wiki can preserve broader historical evidence, recurring errors, successful strategies, rejected proposals, and root-cause analyses. The reported experiments suggest that this separation improves average performance, scaling returns, cross-model transfer, and later-stage refinement. They do not establish that persistent knowledge accumulation is sufficient for safe or stable self-evolution.

WikiSkill therefore defines a framework in which agent capability development is cumulative rather than iteration-local:

$W_k$6

Its distinguishing principle is that skill evolution should preserve not only what an agent currently does, but also why previous procedures succeeded or failed.