AutoSkill: Evolving Reusable Skill Architectures

Updated 5 July 2026

AutoSkill is a family of skill-centric architectures that externalize procedural memory as reusable artifacts, spanning robotics to LLM plugins.
It integrates methods such as autonomous playing, test-time adaptation, and failure-driven evolution to optimize skill retrieval and injection.
Empirical results demonstrate significant improvements in task success and efficiency through adaptive skill synthesis across varied application domains.

AutoSkill denotes a family of skill-centric architectures in which reusable procedures are made explicit as external artifacts—such as SKILL.md files, structured contracts, executable skill folders, or temporary synthesized instructions—and then retrieved, injected, tested, merged, or refined to improve agent or robot behavior without relying solely on parameter updates (Yang et al., 1 Mar 2026). In recent work, the term spans lifelong personalized LLM plugins, test-time adaptive skill synthesis from trajectories, external-knowledge compilation into procedural memory, self-evolving agent repositories, and earlier robotic programming through autonomous playing and skill-centric testing (Wang et al., 16 May 2026, Pan et al., 8 Jun 2026, Hangl et al., 2017). This suggests that AutoSkill is best understood not as a single algorithm, but as a research program centered on explicit, reusable, and evolvable skill representations.

1. Scope and historical development

The earliest AutoSkill formulation in the provided literature is robotic. "A novel Skill-based Programming Paradigm based on Autonomous Playing and Skill-centric Testing" (Hangl et al., 2017) combines user-provided basic behaviour skeletons, autonomous skill acquisition by robotic playing, visual programming, kinesthetic teaching, and experience-based self-diagnosis. The 2026 LLM-oriented AutoSkill literature redefines the term around externalized procedural memory: "AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution" (Yang et al., 1 Mar 2026) derives and maintains skills from dialogue traces; "Skills on the Fly: Test-Time Adaptive Skill Synthesis for LLM Agents" (Wang et al., 16 May 2026) synthesizes temporary task-specific skills at inference time; and "Anything2Skill: Compiling External Knowledge into Reusable Skills for Agents" (Pan et al., 8 Jun 2026) compiles heterogeneous corpora into a persistent SkillBank.

System	Skill source	Primary mechanism
AutoSkill (robotics) (Hangl et al., 2017)	Basic behaviours + autonomous playing	Extend DoA and diagnose failures
AutoSkill (Yang et al., 1 Mar 2026)	Dialogue and interaction traces	Extract, merge, retrieve, version
SkillTTA (Wang et al., 16 May 2026)	Retrieved trajectories	Synthesize temporary textual skill
Anything2Skill (Pan et al., 8 Jun 2026)	External knowledge records	Compile skill contracts into SkillBank
EvoSkill (Alzubi et al., 3 Mar 2026)	Execution failures	Propose and materialize skill folders
MUSE-Autoskill (Lin et al., 26 May 2026)	On-demand creation + feedback	Create, test, refine, store memory

A central continuity across these systems is the shift from latent capability to explicit procedural assets. A central divergence is the locus of adaptation: some methods adapt via context injection into a frozen solver, some via continuous repository maintenance, some via evolutionary selection over skill-bearing programs, and some via embodied demonstration retrieval.

2. Formalizations and representations of a skill

No single formal definition is shared across the literature. In the 2026 lifelong-learning formulation, each skill is a 7-tuple

$s=(n,d,p,\tau,\gamma,\xi,v),$

where $n$ is the name, $d$ the description, $p$ the executable instruction prompt, $\tau$ a trigger set, $\gamma$ a tag set, $\xi$ an example set, and $v$ a version number (Yang et al., 1 Mar 2026). Because skills are stored externally in SKILL.md artifacts, they are editable, inspectable, and versioned.

Anything2Skill generalizes this into a richer contract:

$s=\langle n,d,a,v,g,\tau,\sigma^+,\sigma^-,\mathbf{m},\mathbf{p},\mathbf{c},\mathbf{o},\mathcal{E},\rho\rangle,$

with asset type $a$ , taxonomy node $n$ 0, granularity $n$ 1, invocation and contraindication conditions $n$ 2, action moves $n$ 3, workflow steps $n$ 4, constraints $n$ 5, output specification $n$ 6, supporting evidence $n$ 7, and confidence $n$ 8 (Pan et al., 8 Jun 2026). This representation makes procedural knowledge auditable and evidence-linked.

Folder-based representations appear in EvoSkill and MUSE-Autoskill. EvoSkill stores each skill under skills/s_i/ with SKILL.md, metadata.json, and optional helper code in scripts/, while whole programs are tracked on Git branches with a program.yaml recording parentage and validation score (Alzubi et al., 3 Mar 2026). MUSE-Autoskill materializes skills as packages containing SKILL.md, scripts/, and tests/, and associates each skill with .memory.md that accumulates usage experience (Lin et al., 26 May 2026).

SkillTTA uses a lighter representation: a temporary textual skill $n$ 9 synthesized specifically for a test query $d$ 0, then injected into the solver prompt (Wang et al., 16 May 2026). By contrast, the 2017 robotics paper defines a skill as

$d$ 1

where $d$ 2 is a basic behaviour and $d$ 3 is the success predicate (Hangl et al., 2017). The representational spectrum therefore ranges from abstract prompt artifacts to executable code packages to embodied behavior-success pairs.

3. Retrieval, injection, and lifecycle management

The core 2026 AutoSkill architecture is a training-free plugin layer built from prompt templates and LLM components for query rewriting, skill-conditioned generation, extraction, judgment, merging, and embedding (Yang et al., 1 Mar 2026). At inference time it rewrites the current query, performs hybrid dense-plus-BM25 retrieval over the skill bank, keeps the top- $d$ 4 skills above threshold $d$ 5, renders them into context, and generates the response. Its relevance score is

$d$ 6

with $d$ 7 the normalized dense and BM25 scores. In the background loop, windowed user queries are abstracted into candidate skills, matched to a nearest neighbor, and then assigned one of three management actions: add, merge, or discard.

Anything2Skill extends lifecycle management beyond simple insertion. It first decomposes documents into evidence windows, performs plan-and-expand extraction under a skill-tree prior, compiles drafts into canonical skills via a taxonomy-aware key $d$ 8, and then reconciles them against the global registry using a hybrid similarity function $d$ 9 that combines cosine similarity, BM25, and taxonomy indicators (Pan et al., 8 Jun 2026). Registry actions are richer than in the base AutoSkill system:

$p$ 0

Each skill also carries a lifecycle state

$p$ 1

and updates log both structural deltas and evidence deltas.

MUSE-Autoskill embeds lifecycle management directly into an agent loop. Skill Management retrieves and ranks candidate skills; Skill Creation generates a new skill if none suffices; Skill Evaluation runs tests in a sandbox; Skill Refinement patches failed skills; Skill Memory appends outcomes to .memory.md (Lin et al., 26 May 2026). This is a stronger notion of operational governance than plain retrieval.

A common theme across these systems is that skills are treated as curated procedural memory rather than raw text memory. That distinction is explicit in AutoSkill’s comparison to retrieval-augmented memory systems and in Anything2Skill’s separation of declarative passage retrieval from procedural skill retrieval (Yang et al., 1 Mar 2026, Pan et al., 8 Jun 2026).

4. Test-time adaptation and failure-driven evolution

SkillTTA formulates AutoSkill as test-time adaptation without parameter updates. Given a training pool

$p$ 2

the method embeds metadata $p$ 3 with a text encoder $p$ 4, retrieves the top- $p$ 5 nearest trajectories by cosine similarity,

$p$ 6

synthesizes a temporary textual skill

$p$ 7

and then solves with a frozen solver

$p$ 8

In practice $p$ 9, and ablations report that $\tau$ 0 can introduce noise (Wang et al., 16 May 2026).

The reported gains are substantial. On SpreadsheetBench, Pass@1 rises from 0.397 for Trace2Skill to 0.505 for SkillTTA, while on BigCodeBench Pass@1 rises from 0.517 to 0.651. On ALFWorld, SkillTTA reaches 0.872 success with 8.88 average steps, matching a heavier memory-learning baseline within four points of success rate while producing the shortest successful trajectories among reported methods (Wang et al., 16 May 2026). The ablations are especially informative: on SpreadsheetBench, no retrieval gives 0.410, raw top-3 trajectories 0.422, random retrieval 0.445, and SkillTTA 0.540; failed-only top-3 yields 0.540, success-only top-3 0.532, and all top-3 0.505. This directly supports the claim that synthesized procedural abstraction is not equivalent to raw trajectory prompting.

EvoSkill treats AutoSkill as iterative failure analysis over frozen-base programs. A program $\tau$ 1 is a frozen base LLM plus a set of skills, and EvoSkill searches for high-performing programs while controlling skill complexity through a Pareto-frontier formulation (Alzubi et al., 3 Mar 2026). Its loop repeatedly collects failures, asks a Proposer agent for a new skill or edit, uses a Skill-Builder agent to materialize the proposal, evaluates the child program on a held-out validation set, and retains only improving candidates in a bounded frontier $\tau$ 2. On OfficeQA, exact-match accuracy increases from 60.6% to 67.9%; on SealQA, from 26.6% to 38.7%; and a skill evolved on SealQA improves BrowseComp from 43.5% to 48.8% zero-shot (Alzubi et al., 3 Mar 2026).

SkillCraft provides a benchmark-level corroboration for this line of work. Its evaluation protocol equips agents with list_skills, execute_skill, save_skill, and get_skill, plus a coding verifier. Across 126 tasks, skill saving and reuse reduce token usage by up to 80%, and success rate correlates strongly with tool composition ability at test time, with $\tau$ 3 and $\tau$ 4 (Chen et al., 28 Feb 2026). This suggests that test-time skill abstraction is not merely a compression device; it is closely coupled to task success.

5. Embodied and model-native extensions

The robotic AutoSkill line predates the LLM work and gives the term a broader meaning. In the 2017 framework, a user first supplies a basic behaviour $\tau$ 5, after which the robot performs a stochastic random walk through an Episodic & Compositional Memory graph containing sensing-action clips, perceptual-state clips, preparatory-behaviour clips, and the target basic behaviour (Hangl et al., 2017). The transition update is

$\tau$ 6

with binary reward $\tau$ 7 derived from the success predicate. The paper reports that Book Grasping, initially valid for one orientation, expands after approximately 300 playing episodes to 8 discrete book rotations with average success $\tau$ 8; Tower Disassembly achieves $\tau$ 9 success on 2- and 3-box towers; and skill-centric testing identifies a changed planner module as top suspect within 5 diagnostic trials (Hangl et al., 2017).

Uni-Skill transfers the self-evolving repository idea into generalizable robotic manipulation. Its planner uses a Sufficiency Discriminator $\gamma$ 0, a Skill Generator $\gamma$ 1, and a Planner $\gamma$ 2 over a base API library plus generated skills (Xie et al., 3 Mar 2026). When a required skill is absent, Uni-Skill retrieves examples from SkillFolder, a four-level hierarchy $\gamma$ 3– $\gamma$ 4 inspired by VerbNet, built from automatically annotated robot videos. The repository contains over 10,000 skill-aligned clips from 350 h of DROID, organized into 106 VerbNet classes and 1,659 unique descriptions (Xie et al., 3 Mar 2026). Retrieved clips provide semantic constraints and trajectory references, which are converted into executable trajectories through a visual-prompting operator. Zero-shot results are reported as 0.42 average success on 8 RLBench tasks covered by $\gamma$ 5, 0.41 on 10 out-of-base tasks versus MOKA 0.10 and CaP approximately 0.01, and 0.73 average success on 8 real-world Franka tasks (Xie et al., 3 Mar 2026).

A more radical extension is the model-native view of skills. "Characterizing Model-Native Skills" (Kang et al., 19 Apr 2026) argues that skill characterization should be grounded in the model’s own activation space rather than external textual ontologies. It forms sequence-level activations $\gamma$ 6, stacks them into an activation matrix $\gamma$ 7, and uses PCA so that the top right-singular vectors $\gamma$ 8 define orthogonal skill directions. The paper reports that 5–10 PCs often capture more than 90% of activation variance for long-form reasoning tasks, and that these directions support both post-training data selection and inference-time steering. Reported gains include Pass@1 improvements of up to 20% on MATH and 41% on AMC, and Pass@8 gains of up to 4.8% on MATH (Kang et al., 19 Apr 2026). This broadens the notion of AutoSkill from explicit artifacts to latent but intervention-ready axes of behavioral variation.

6. Empirical record, transfer behavior, and limitations

Across the literature, AutoSkill systems are evaluated on markedly different substrates—dialogue logs, trajectories, knowledge corpora, tool APIs, and robot videos—yet they repeatedly report gains from explicit proceduralization.

Approach	Benchmark(s)	Reported outcome
SkillTTA (Wang et al., 16 May 2026)	SpreadsheetBench / BigCodeBench / ALFWorld	0.397→0.505 Pass@1; 0.517→0.651 Pass@1; 0.872 SR with 8.88 steps
Anything2Skill+RAG (Pan et al., 8 Jun 2026)	qsv / GitHub-CLI	98.85% / 94.10% success
EvoSkill (Alzubi et al., 3 Mar 2026)	OfficeQA / SealQA / BrowseComp	60.6→67.9; 26.6→38.7; 43.5→48.8 zero-shot
MUSE-Autoskill (Lin et al., 26 May 2026)	SkillsBench	53.19%→60.35% with self-created skills; 87.94% on 35 covered tasks
SkillCraft (Chen et al., 28 Feb 2026)	126 tool-use tasks	Up to 80% token savings
Uni-Skill (Xie et al., 3 Mar 2026)	RLBench / Franka	0.42 base-covered RLBench; 0.41 out-of-base; 0.73 real-world
Model-native skills (Kang et al., 19 Apr 2026)	MATH / AMC	Pass@1 up to 20% and 41%; Pass@8 +4.8%

Several patterns recur. First, skills are repeatedly shown to be more useful than raw evidence alone. SkillTTA reports 0.540 for synthesized skills versus 0.422 for raw top-3 trajectories on SpreadsheetBench, and Anything2Skill explicitly positions procedural skills as complementary to RAG’s declarative passages (Wang et al., 16 May 2026, Pan et al., 8 Jun 2026). Second, transfer is a persistent theme: EvoSkill reports zero-shot BrowseComp gains from SealQA-evolved skills, SkillCraft reports cross-task and cross-model reuse, and MUSE-generated skills improve Hermes from 47.89 to 58.40 without human-authored skills (Alzubi et al., 3 Mar 2026, Chen et al., 28 Feb 2026, Lin et al., 26 May 2026). Third, explicit governance matters. MUSE evaluates skills with tests and runtime feedback, Anything2Skill tracks lifecycle states and versions, and AutoSkill emphasizes Git-friendly, shareable artifacts rather than opaque internal weights (Lin et al., 26 May 2026, Pan et al., 8 Jun 2026, Yang et al., 1 Mar 2026).

The limitations are equally consistent. SkillTTA reports that larger retrieval sets can introduce noise, with SpreadsheetBench performance declining from 0.540 at $\gamma$ 9 to 0.519 at $\xi$ 0 and 0.497 at $\xi$ 1 (Wang et al., 16 May 2026). EvoSkill notes computational cost, single-seed evaluation, the absence of vision or multimodal skills, and the fact that frontier admission is implemented only on validation score rather than true Pareto dominance over performance and complexity (Alzubi et al., 3 Mar 2026). SkillCraft observes that hierarchical composition preserves high execution rates but can degrade overall success because of cascading-error propagation; for GPT-5.2, flat Skill mode reaches 90% SR while Hierarchical Mode drops to 79% (Chen et al., 28 Feb 2026). Anything2Skill recommends human-in-the-loop overrides for merges and splits, monthly SkillBank versioning, and confidence monitoring, indicating that repository governance remains an open systems problem rather than a solved retrieval problem (Pan et al., 8 Jun 2026).

Taken together, the literature portrays AutoSkill as an attempt to elevate “skill” into a first-class computational object: explicit enough to retrieve, test, merge, and transfer; compact enough to inject at inference time; and structured enough to outlive a single prompt, episode, or session. The main unresolved question is not whether skills can be externalized, but which representation—textual artifact, executable folder, trajectory-conditioned prompt, hierarchical repository entry, or activation-space direction—best captures reusable capability for a given domain.