--- title: 'AgentSkillOS: Skill Organization & Execution' url: https://www.emergentmind.com/topics/agentskillos type: topic --- # AgentSkillOS: Skill Organization & Execution AgentSkillOS is a framework for managing and using very large ecosystems of agent skills. It is presented as the first principled system for three tightly coupled problems: skill organization at ecosystem scale, skill selection for a particular user task, and multi-skill orchestration and execution so an agent can solve tasks that exceed the capability of any single skill [2603.02176]. In the motivating setting of rapidly expanding Claude-style agent skills, modular packages containing instructions, scripts, and auxiliary resources that can be dynamically loaded at runtime, the central claim is that structured composition is essential: capability-tree retrieval can closely approximate oracle skill selection, and DAG-based orchestration significantly outperforms flat or native skill invocation even when both systems are given the same skills [2603.02176]. ## 1. Definition, scope, and ecosystem setting AgentSkillOS separates the problem into two stages: **Manage Skills** and **Solve Tasks** [2603.02176]. The first stage organizes a large skill ecosystem into a hierarchical capability tree for efficient discovery; the second retrieves relevant skills from that tree and composes them into an executable DAG of subtasks and dependencies [2603.02176]. The framework is motivated by the rapid expansion of Claude-style skill ecosystems, for which the paper reports **280,000+ public skills** by late February 2026 [2603.02176]. The framework’s problem statement is that skill ecosystems become difficult to use as they grow because skills are too numerous for users to understand globally, heterogeneous in naming, quality, and overlap, decentralized across many third-party contributors, and often fragmented and isolated, lacking explicit mechanisms for composition [2603.02176]. AgentSkillOS therefore treats the bottleneck not as mere skill availability, but as the absence of an intermediate systems layer that can organize, retrieve, and compose skills structurally [2603.02176]. This framing aligns with broader contemporaneous work that treats skills as an infrastructure layer rather than as isolated prompts. The empirical analysis in “Agent Skills: A Data-Driven Analysis of Claude Skills for Extending Large Language Model Functionality” formalizes a skill as \[ \text{Skill} = \{\text{Metadata}, \text{Instructions}, \text{Resources}\}, \] and reports a public marketplace snapshot of **40,285** skills with substantial redundancy and non-trivial safety exposure [2602.08004]. This suggests that AgentSkillOS is best understood not simply as a retrieval method, but as an attempt to impose systems structure on a rapidly scaling and already heterogeneous skill economy. ## 2. Core architecture: capability-tree management and DAG orchestration The offline management stage constructs a capability tree over a managed skill subset. Let the full skill ecosystem be \( \mathcal{S} \), the active subset \( \mathcal{S}_T \), and the capability tree \(T\) [2603.02176]. Each node \(n \in T\) corresponds to a partition \( \mathcal{S}_n \subseteq \mathcal{S}_T \), with root set \[ \mathcal{S}_r = \mathcal{S}_T, \] and child partitions satisfying \[ \mathcal{S}_n=\bigcup_{c\in \mathrm{ch}(n)}\mathcal{S}_c \] and \[ \mathcal{S}_c\cap \mathcal{S}_{c'}=\emptyset \quad \text{for any } c\neq c'. \] Leaves ultimately correspond to individual skills [2603.02176]. The intended behavior is coarse-to-fine localization: broad capability regions are identified first, then refined into specific leaves [2603.02176]. Tree construction is breadth-first and uses recursive node-level categorization. For each node, an LLM first performs **Group Discovery**, generating category groups with names and descriptions, and then **Skill Assignment**, assigning each skill to one discovered category [2603.02176]. At the root, AgentSkillOS does not rely on unconstrained discovery; it fixes five top-level groups manually: **content creation**, **data processing**, **software development**, **automation**, and **domain-specific** [2603.02176]. The active metadata for tree construction are limited to skill **name** and **description** [2603.02176]. The paper gives concrete tree hyperparameters. For ecosystems of size **200** and **1K**, the branching factor is \[ B = 7. \] For the **200K** ecosystem, \[ B = 12. \] Child groups are discovered in the range \[ [B-3,\; B+2], \] and the per-node capacity threshold is \[ C = \lfloor 1.5B \rfloor. \] Thus, when \(B=7\), \(C=10\); when \(B=12\), \(C=18\) [2603.02176]. To limit active-tree size at very large scale, the framework introduces a usage-frequency queue \(Q\), where the frequency \(f(s)\) is the skill’s **install count** on the marketplace [2603.02176]. The active set is defined as \[ \mathcal{S}_T=\mathrm{TopK}(Q,K)\cup \mathcal{S}^{\mathrm{user}}, \] where \( \mathrm{TopK}(Q,K) \) selects the most frequently used skills and \( \mathcal{S}^{\mathrm{user}} \) injects manually selected user skills [2603.02176]. For the **200K** ecosystem, the paper sets \[ K = 10{,}000. \] The remainder is placed in a dormant index, which uses a vector index over skill name and description for semantic suggestion via embedding similarity [2603.02176]. The online task-solving stage begins with task-driven tree traversal. The LLM descends the hierarchy layer by layer, selecting relevant category nodes and collecting reached leaves as candidate skills [2603.02176]. Candidate skills are then pruned by an LLM that deduplicates and ranks them, keeping the top \[ M = 8 \] shortlisted skills [2603.02176]. These shortlisted skills define the node set \(V\) of the orchestration graph \[ G = (V,E), \] where each \(v \in V\) is a selected skill and each edge \((u,v)\in E\) denotes a dependency [2603.02176]. The graph must satisfy the layering constraint \[ (u,v)\in E \Rightarrow \ell(u)<\ell(v), \] which enforces acyclicity through topological layering [2603.02176]. The orchestration layer generates alternative DAG plans under three prompt-level strategies: **Quality-First**, **Efficiency-First**, and **Simplicity-First** [2603.02176]. Quality-First adds preparation and refinement stages to maximize output quality; Efficiency-First reduces sequential dependencies and exposes more parallelism; Simplicity-First produces a minimal DAG in which every node is essential [2603.02176]. Execution follows the DAG’s layered structure: nodes in the same layer run in parallel, while different layers run sequentially according to dependency order [2603.02176]. Each execution prompt includes the original user task, the specific skill to invoke, the assigned subtask, upstream artifacts, usage hints for those artifacts, expected outputs, and downstream-consumption explanations [2603.02176]. ## 3. Benchmark design and evaluation methodology AgentSkillOS is evaluated on a benchmark of **30 artifact-rich tasks** spanning **five categories**, with **6 tasks per category**: **Data Computation**, **Document Creation**, **Motion Video**, **Visual Design**, and **Web Interaction** [2603.02176]. The benchmark is designed to test whether an agent can discover relevant skills, invoke them correctly, compose them when needed, and deliver end-user-facing artifacts such as **PDF**, **PPTX**, **DOCX**, **HTML pages**, **videos**, **generated images**, **csv**, **json**, and **gif** [2603.02176]. All tasks are human-crafted by experts, who curate high-quality skills from public marketplaces and GitHub repositories and then write task descriptions and deliverable requirements based on the scenarios those skills target and plausible real-world user needs [2603.02176]. Some tasks are derived from a single skill, while others are created by cross-composing multiple skills [2603.02176]. This design intentionally makes success strongly dependent on correct skill selection and composition [2603.02176]. Evaluation uses pairwise LLM-based judging rather than absolute scoring. The judge is implemented using **Claude Code Agent SDK with claude-opus-4.5** [2603.02176]. Because outputs are multimodal, the benchmark first converts them into judge-consumable representations: documents and slides are rendered as page images, HTML pages as full-page screenshots, videos as uniformly sampled frames plus duration, resolution, and frame rate metadata, images are resized to a standard resolution, and text files are included verbatim up to a length limit [2603.02176]. To reduce position bias, every pairwise comparison is run in both orderings; if both orderings agree, that preference is used, if one ordering errors the valid one is used, and if orderings conflict the result is recorded as a tie [2603.02176]. Results are aggregated into a win matrix \[ W \in \mathbb{R}^{N \times N}, \] where \(W_{ij}\) counts how many times system \(i\) is preferred over system \(j\), with ties contributing \(0.5\) to both directions [2603.02176]. Final ranking uses a Bradley–Terry model with latent strengths \( \beta_i \in \mathbb{R} \), using the standard probability \[ P(i \succ j) = \frac{e^{\beta_i}}{e^{\beta_i} + e^{\beta_j}}, \] fit by maximum likelihood with the **MM algorithm** and **Laplace smoothing** \( \alpha = 1 \) [2603.02176]. Strengths are centered and linearly rescaled to \([0,100]\) through \[ S_i = \frac{\beta_i - \beta_{\min}}{\beta_{\max} - \beta_{\min}} \times 100. \] This \(S_i\) is the paper’s unified quality score [2603.02176]. The experimental setup evaluates three ecosystem scales: - \( |\mathcal{S}| = 200 \) - \( |\mathcal{S}| = 1\text{K} \) - \( |\mathcal{S}| = 200\text{K} \) The **200-skill** pool is manually curated from the best-performing skill for each benchmark task plus additional expert-selected skills; the **1K** and **200K** pools extend this base with more marketplace skills ranked by install count [2603.02176]. AgentSkillOS uses **claude-opus-4.5** for capability tree construction, retrieval, and DAG planning, while DAG node execution uses **Claude Code Agent SDK with claude-sonnet-4.5** [2603.02176]. ## 4. Empirical findings The main result is that AgentSkillOS variants dominate flat or skill-free baselines across all three ecosystem scales [2603.02176]. At \( |\mathcal{S}| = 200 \), the Bradley–Terry scores are: **Quality-First 100.0**, **Efficiency-First 58.5**, **Simplicity-First 53.6**, **w/ Full Pool 24.3**, and **Vanilla 0.0** [2603.02176]. At \( |\mathcal{S}| = 1\text{K} \), they are **100.0**, **76.1**, **56.3**, **48.1**, and **0.0**, respectively [2603.02176]. At \( |\mathcal{S}| = 200\text{K} \), they are **100.0**, **89.0**, **56.0**, **17.2**, and **0.0** [2603.02176]. A particularly important finding is that exposure to a larger flat skill pool does not scale gracefully. The **w/ Full Pool** baseline scores **24.3** at 200 skills, **48.1** at 1K, and then drops to **17.2** at 200K [2603.02176]. This is the empirical basis for the paper’s claim that skill potential is unlocked by structured composition rather than by skill availability alone [2603.02176]. The ablation studies isolate both retrieval and orchestration. Tree-based retrieval is said to approach oracle selection: the gap between **Quality-First** and **Quality-First (Oracle)** is only modest and narrows as ecosystem size grows [2603.02176]. More importantly, **AgentSkillOS Quality-First** still clearly outperforms **w/ Oracle Skills**, meaning flat invocation underperforms structured DAG orchestration even when the exact benchmark-designated skills are already known [2603.02176]. This identifies orchestration, not only retrieval, as a primary causal ingredient. The three orchestration strategies also induce measurably different graph topologies. **Quality-First** produces the largest, deepest, most connected DAGs; **Efficiency-First** produces similarly sized but wider and shallower DAGs; **Simplicity-First** produces the smallest and sparsest DAGs [2603.02176]. The paper analyzes graph structure using node count, edge count, max width, and max depth, though it does not report exact graph-statistic values in the provided text [2603.02176]. The empirical results fit a broader trend in adjacent systems work. In “AgentStore,” heterogeneous-agent integration improves OSWorld performance from **11.21%** to **23.85%** average task success using a registry, manifest, and meta-controller design [2410.18603]. In “Bian Que,” a domain-specific skill arrangement framework for online operations reports **99.0%** pass rate on offline evaluations after refinement and deployment outcomes including **75%** alert-volume reduction, **80%** RCA accuracy, and more than **50%** MTTR reduction [2604.26805]. These systems are not AgentSkillOS itself, but they reinforce the claim that structured skill organization, routing, and orchestration are high-leverage systems variables rather than secondary conveniences. ## 5. Relation to adjacent architectures and security models AgentSkillOS, as defined in [2603.02176], focuses on large-scale skill organization and orchestration. Nearby papers illuminate adjacent subsystems that a broader AgentSkillOS stack might incorporate. AgentClick contributes a reusable **review plane** rather than a full operating system design. It defines a localhost/HTTP service, a browser UI, and a skill-mediated protocol through which an otherwise unmodified terminal agent can surface artifacts for inspection, accept edits, persist preferences, and resume execution [2604.16520]. Its core event loop is explicitly described as **proposal → review session → human action → result signal → agent resumes**, which the paper itself characterizes, in AgentSkillOS terms, as a generic human-review syscall or interrupt path [2604.16520]. This suggests a practical supervision substrate for skill execution, especially at consequential steps. AgentStore contributes a platform model for integrating heterogeneous agents as skills. It defines **AgentPool**, **AgentEnroll**, and **MetaAgent**, with an enrollment representation \[ \mathcal{A} = \{(a_1,d_1), (a_2,d_2), ..., (a_n,d_n)\}, \] where each \(d_i\) is a standardized document describing applications, capabilities, limitations, and demonstrations [2410.18603]. It also proposes **AgentToken**, a learned token-based indexing layer for scalable routing, with next-token prediction over the union of vocabulary and agent tokens: \[ P_M(t_i | t_{