SkillX: Automated SkillKB Framework

• SkillX is a fully automated framework that constructs hierarchical skill knowledge bases to improve generalization and sample efficiency in LLM agents.
• It extracts three levels of skills—planning, functional, and atomic—using iterative extraction, refinement, and merging techniques from agent trajectories.
• The system integrates robust retrieval methods and an open API to inject structured skills into agents, boosting both in-distribution and zero-shot performance.

SkillX is a fully automated framework for constructing skill knowledge bases (SkillKBs) aimed at improving the generalization, sample efficiency, and transfer performance of LLM agents in complex, long-horizon, user-interactive environments. SkillX (also referenced as AutoSkills) emphasizes the automatic distillation of agent trajectories into reusable and hierarchically structured skills without the reliance on domain engineering or manual curation. A SkillKB constructed by SkillX can be injected into a wide class of LLM-based agents, enhancing both in-distribution and zero-shot performance by supplying explicit, structured knowledge representations that directly guide tool utilization, planning, and low-level execution (Wang et al., 6 Apr 2026).

1. Hierarchical Skill Abstraction

SkillX encodes agent procedural knowledge into a three-level hierarchy:

• Planning Skills ($S_\text{plan}$): High-level task decompositions that express top-level strategies as ordered lists of sub-tasks, typically abstracted from entire successful trajectories.
• Functional Skills ($S_\text{func}$): Reusable subroutines (macros) mapping to semantically coherent subtasks, usually associated with a small set of functionally related tool invocations and parameter flows.
• Atomic Skills ($S_\text{atomic}$): Fine-grained, single-tool usage patterns capturing canonical behaviors, parameter conventions, and invocation constraints.

Given a successful trajectory $\tau = (o_1, a_1, \dots, o_T, a_T)$, SkillX extracts these skills using three corresponding functions $f_\text{plan}(\tau)$, $f_\text{func}(\tau, s_\text{plan})$, and $f_\text{atomic}(\tau, \text{Tools})$. Each extracted skill is represented as a JSON-like object with fields for name, document (Markdown format), content (code or pseudocode), tools (associated tool names), and an embedding vector for retrieval.

2. Automated Pipeline Design

The SkillX system operates as an iterative, fully automated pipeline consisting of three synergistic components:

2.1 Multi-Level Skills Design

• Extraction: The pipeline begins with collecting agent rollouts on a suite of training tasks. From successful trajectories, $S_\text{plan}$ are extracted by prompting the LLM backbone to generate sub-task lists. These plans are used as guides for extracting $S_\text{func}$, which in turn guide the extraction of $S_\text{atomic}$ per tool instance in the trajectory.
• Each level’s extraction uses LLM prompts tailored to remove exploratory/backtracking steps, merge repetitive patterns, and capture reusable behaviors.

2.2 Iterative Skills Refinement

• Skill Merge: Candidate skills are clustered using embedding cosine similarity (threshold $S_\text{func}$0) and merged via LLM prompts that synthesize a representative skill covering all in-cluster variations. The merged skill’s embedding is updated by aggregating per-skill vector deviations: for a centroid $S_\text{func}$1, the update direction is $S_\text{func}$2.
• Skill Filter: General filtering removes non-standard-library dependencies, "thin wrappers," environment-anchored logic, or hardcoded parameters. Tool-specific filters validate that generated “content” fields are schema-conformant and contain neither parameter nor ordering errors, enforced by an LLM-based checker.

2.3 Exploratory Skills Expansion

• Tool-Guided Exploration: To address limited tool coverage in seed data, SkillX samples underused, failure-prone, and low-confidence tools based on a scoring weight $S_\text{func}$3. Synthetic tasks are generated to trigger these tools, and resulting successful trajectories yield new skills, which are then subjected to merge/filter.
• The library grows additively with each iteration: $S_\text{func}$4.

3. Retrieval and Agent Integration

At inference, SkillX augments a base agent’s execution by dynamically retrieving relevant skills from the SkillKB:

• Retrieve the most relevant planning skills: $S_\text{func}$5, where $S_\text{func}$6 is a hybrid semantic retriever over the skill index.
• Use retrieved plans to rewrite the pseudo-plan for task $S_\text{func}$7 (via LLM prompts).
• For each plan step, retrieve functional and atomic skills with MMR selection, top-k ($S_\text{func}$8), and similarity/deduplication thresholds ($S_\text{func}$9 and within $S_\text{atomic}$0 of top score, deduplicate if $S_\text{atomic}$1).
• All retrieved skills and the generated plan are injected into the agent system prompt in one pass before execution, increasing guidance without requiring parameter updates to the backbone.

4. Implementation and Technical Details

• Backbone agent: GLM-4.6 is used for both trajectory generation and LLM-based skill extraction, plan rewriting, and execution; Qwen3-Embedding-8B provides embedding vectors.
• Indexing: Skills are indexed via FAISS (HNSW) for k-NN retrieval.
• API: The SkillKB Python API supports extraction, refinement, skill expansion, and semantic retrieval. All skills store their document, code, associated tools, and embedding vector.
• Retrieval: The system retrieves up to 100 candidates for initial filtering and reduces via thresholds and diversity objectives.
• Prompting configuration: Extraction is performed at temperature 0.9; exploration uses temperature 1.0.

5. Empirical Evaluation and Benchmarks

SkillX has been evaluated on three complex, long-horizon interactive benchmarks: BFCL-v3, AppWorld, and $S_\text{atomic}$2-Bench (Retail, Airline, Telecom). Results for Qwen3-32B, Kimi-K2-Instruct, and GLM-4.6 show consistent and substantial lifts in both Avg@4 and Pass@4 metrics relative to non-memory and experience-only (“ExpeL”) baselines.

Model	Method	BFCL-v3 Avg@4	AppWorld Avg@4	τ²-Bench Retail
Qwen3-32B	No Memory	53.7	27.7	53.8
	ExpeL	57.3	32.9	56.3
	AutoSkills	63.7	35.1	66.9
Kimi-K2-Inst-09	No Memory	65.2	46.9	75.6
	AutoSkills	66.8	56.4	78.1
GLM-4.6	No Memory	76.7	60.3	76.3
	AutoSkills	79.5	64.9	82.5

Average token usage is reduced by 10–15% and execution steps by 20–30% relative to No-Memory baselines. Qualitative traces demonstrate that skill-guided agents complete long-horizon user-interactive tasks with greater sample efficiency and higher success on first attempt (Wang et al., 6 Apr 2026).

6. Illustrative Skill Examples

• Planning Skill: Decomposes a high-level request (e.g., "Add track via collaborator filter") into sequential steps: retrieve playlists, filter by collaborator, add track.
• Functional Skill: Encodes macros such as spotify_add_track_to_playlist that check existence and perform retry logic (see original content for pseudocode).
• Atomic Skill: Encapsulates canonical tool patterns (e.g., paginated playlist fetch).

These structured representations enable fine-grained, composable, and semantically motivated guidance at multiple levels in an agent’s decision process.

7. Broader Significance and Context

SkillX integrates hierarchical, modular skill abstractions with fully automated extraction, iterative refinement, and exploration-driven coverage expansion. The approach yields transferable, plug-and-play SkillKBs that consistently improve both average performance and zero-shot generalization across diverse LLM agents. Empirical analyses show systematic gains in both efficiency and robustness, with notable strengths in reducing redundant exploration and providing explicit, verifiable execution guidance (Wang et al., 6 Apr 2026).

The SkillX methodology is complementary to dual-stream “experience+skill” approaches such as XSkill (Jiang et al., 12 Mar 2026), and runtime-enforced skill abstractions (“Formal Skill”) as in FairyClaw (Zhang et al., 19 May 2026). A plausible implication is that direct skill injection, when combined with structured refinement and coverage-driven expansion, will be a persistent theme in emergent agent systems. The modular open API and experience-integrated design further facilitate transfer and reuse under varied agent architectures and task distributions.