---
title: Structured Skill Composition
url: https://www.emergentmind.com/topics/structured-skill-composition
type: topic
---

# Structured Skill Composition

Structured skill composition denotes a family of methods that make the combination of reusable skills explicit rather than leaving it implicit in a monolithic policy, prompt, or retrieved text block. In the surveyed literature, the composed objects range from policy embeddings and language skills to LoRA modules, graph nodes, scene-graph-conditioned robot primitives, and reward-evaluation procedures. The common aim is to solve new tasks by organizing previously available capabilities through typed interfaces, dependency structure, factorization, or evidence-bearing intermediate representations rather than by relearning each task from scratch [1711.11289][2308.00304][2502.05932][2604.17870][2606.03980][2606.08169].

## 1. Definitions and scope

The term *skill* is used heterogeneously across the literature, and the meaning of composition depends on that choice. In ComposeNet, a skill is a policy module represented by a skill-state embedding, and composition is a trainable map
\[
C:\langle \iota_e^{(1)}(S),\iota_e^{(2)}(S),\ldots,\iota_e^{(n)}(S)\rangle \to \iota_c(S),
\]
so the output of composition remains in the same embedding space and can be recursively reused [1711.11289]. In Skills-in-Context prompting and in SKILL-MIX-style fine-tuning, skills are named language capabilities such as rhetorical, literary, reasoning, or theory-of-mind constraints, and composition means producing a single coherent text that simultaneously satisfies a requested set of skills [2308.00304][2409.19808]. In PSEC, a skill primitive is a LoRA update \(\Delta W_i=B_iA_i\) attached to a frozen base policy, so composition occurs directly in parameter space [2502.05932]. In CLASP, a skill is the pair \((\mathcal{O},\phi)\), where \(\mathcal{O}\) is a TP-KMP and \(\phi\) is a VLM-generated schema [2606.08169].

Agent papers make the interface even more explicit. SkillComposer defines a skill as
\[
s_i=(m_i,C_i,\pi_i,T_i,R_i),
\]
with metadata, applicability condition, procedural policy, termination condition, and optional callable interface or support resource; composition is then the prediction of an ordered skill sequence over a fixed library [2606.32025]. Skill-RM adopts the abstract form \(S=(M,U)\) and specializes it to a Reward-Evaluation Skill
\[
\mathcal{S}_{\mathrm{RM}}=(\mathcal{M}_{\mathrm{RM}},\mathcal{U}_{\mathrm{RM}}),
\]
where composition concerns evaluative resources such as rubrics, references, checklists, verifiers, and calibration rules rather than task-execution policies [2606.03980].

This variation implies that structured skill composition is not a single architecture class. In the surveyed work, it includes latent neural composition, in-context decomposition, graph-structured orchestration, parameter-space merging, factor-wise latent control, trajectory-level probabilistic fusion, and even composition specialized to evaluation or safety analysis. A plausible implication is that the field is best understood through the structure imposed on the composition process—shared latent interfaces, typed graphs, factorized variables, task-parameterized frames, or evidence objects—rather than through any single definition of “skill.”

## 2. Representational forms

The literature differs chiefly in what it chooses as the compositional unit and what explicit structure it places around that unit.

| Work | Skill unit | Explicit structure |
|---|---|---|
| "Learning to Compose Skills" [1711.11289] | Skill-state embedding | Binary recursive composition in a shared latent space |
| "Skills-in-Context Prompting" [2308.00304] and "Can Models Learn Skill Composition from Examples?" [2409.19808] | Named language skills | Skill block plus compositional exemplars; synthetic multi-skill examples |
| "Skill Expansion and Composition in Parameter Space" [2502.05932] and "SUSD" [2602.01619] | LoRA modules or factor-wise latent skills | Weighted parameter merging; factorized skill space \(\mathcal{Z}_1\times\dots\times\mathcal{Z}_N\) |
| "GraSP" [2604.17870], "SkillGraph" [2605.12039], and "Generative Skill Composition for LLM Agents" [2606.32025] | Reusable agent skills | Typed DAGs, evolving directed skill graphs, or closed-vocabulary ordered skill sequences |
| "Compose by Focus" [2509.16053], "BrickCraft" [2605.07605], and "CLASP" [2606.08169] | Robot manipulation skills | Focused scene graphs, relation-parameterized assembly plans, and TP-KMP skill schemas |
| "Skill-RM" [2606.03980] and "From Skill Text to Skill Structure" [2604.24026] | Evaluative or documentary skills | Evidence-bearing judgments and Scheduling-Structural-Logical skill structure |

These representations solve different bottlenecks. Shared latent spaces support recursive closure; prompt-grounded named skills support interpretable reasoning traces; graphs expose dependency and repair structure; factorization exposes controllable subspaces; task-parameterized frames expose geometric invariants; and evidence-bearing or source-grounded structures expose auditability. This suggests that “structure” functions as an externalization device: knowledge that would otherwise remain buried in weights or prose is lifted into objects that can be routed, checked, fused, or inspected.

## 3. Composition mechanisms

The mechanisms used to compose skills are correspondingly diverse. ComposeNet trains a differentiable composition function over pretrained skill embeddings and reuses the shared policy layer to decode both primitive and composed behaviors; because the composed embedding has the same type as a primitive embedding, the method supports recursive hierarchies such as \((\neg g \wedge \neg b)\,\mathcal{U}\,r\) and \(\neg b\,\mathcal{U}\,(r\vee g)\) [1711.11289]. SKiC composes by placing a basic-skill block and a compositional exemplar block in the same prompt context, while the SKILL-MIX follow-up paper induces a “meta-skill” by fine-tuning on curated \(k=2,3\) compositions and testing on higher-order or held-out-skill compositions [2308.00304][2409.19808]. Countdown analysis represents solutions as expression trees, treats subtrees as reusable skills, and studies RL post-training as the acquisition of structure-sensitive subtree reuse rather than mere length extension [2512.01775].

Agent systems increasingly formulate composition as an explicit structured prediction problem. SkillComposer predicts
\[
\hat{\mathbf{z}}=(\hat z_1,\hat z_2,\ldots,\hat z_{\hat n},STOP)=f_\theta(x,c,\mathcal S),
\]
so subset, count, and order emerge jointly from a constrained autoregressive decoder over skill identifiers [2606.32025]. GraSP inserts a compilation layer between retrieval and execution, producing an executable typed DAG
\[
G=(V,E),\qquad E\subseteq V\times\{state,data,order\}\times V,
\]
and then performs node-level verification and locality-bounded repair through Rebind, InsertPrereq, Substitute, Rewire, and Bypass [2604.17870]. SkillGraph instead maintains an evolving directed graph with prerequisite, enhancement, and co-occurrence edges, retrieves a task-conditioned subgraph, and topologically sorts it to guide execution [2605.12039].

Parameter-space and factorized methods push composition into the policy representation itself. PSEC merges LoRA skill modules through
\[
W(\theta)=W_0+\sum_{i=1}^{k}\alpha_i(s;\theta)\Delta W_i,
\]
with \(\alpha_i(s;\theta)\) acting as a context-aware routing module [2502.05932]. SUSD factorizes both state and skill space,
\[
\mathcal{S}=\mathcal{S}_1\times\dots\times\mathcal{S}_N,\qquad
\mathcal{Z}=\mathcal{Z}_1\times\dots\times\mathcal{Z}_N,
\]
and trains a single low-level policy \(\pi(a\mid s,z)\) on concatenated factor-wise latents, making composition a property of the representation rather than a separate symbolic operator [2602.01619].

Robotics papers emphasize compositional execution under geometric or relational structure. Compose by Focus builds a focused sub-scene graph over task-relevant objects and relations, encodes it with a two-layer GAT, and conditions a diffusion policy on the resulting graph embedding and skill-language description [2509.16053]. BrickCraft reduces assembly to a sequence of relative operations \(\eta=(b_{\mathrm{ref}},b_{\mathrm{tgt}},\tau)\) and chains pick and assembly skills under explicit preconditions and postconditions [2605.07605]. CLASP composes two selected local KMPs by covariance-weighted Gaussian fusion and requires temporal dominance regions satisfying the compatibility condition in Eq. (1), so fusion is allowed only when one primitive is clearly more certain than the other in each region [2606.08169].

The same pattern appears in task synthesis and reward evaluation. EvoTD defines a skill crossover operator
\[
\mathcal{X}_{\mathrm{skill}}:2^{\mathcal S}\rightarrow\mathcal T
\]
to generate tasks with novel skill combinations and pairs it with parametric mutation over complexity attributes and a dynamic ZPD filter [2605.11666]. Skill-RM executes a reward-evaluation trace
\[
\tau=(a_1,o_1,\ldots,a_T,o_T,z)\sim \pi_\phi(\cdot\mid x,Y;\mathcal S_{\mathrm{RM}})
\]
and reads out reward deterministically from the completed evidence-bearing judgment rather than from a raw scalar head [2606.03980].

## 4. Empirical patterns across domains

Prompt-based and example-based studies show that explicit composition structure can materially improve systematic generalization. In SKiC, with only two compositional exemplars, text-davinci-003 reached 98.5 on 7-digit addition, ChatGPT reached 100.0 on the same setting, ChatGPT reached 92.0 on dynamic-programming length 8, and GPT-4 reached 98.0 on that dynamic-programming setting; on MATH, SKiC raised GPT-4 from 50.3 with ComplexCoT to 56.4 [2308.00304]. In the SKILL-MIX-style fine-tuning study, D1-only training improved single-skill competence but not composition, whereas D2 and D3 training improved unseen higher-order composition and transfer to held-out skill groups; after D1∪D2∪D3 fine-tuning on held-out skills/topics, Mistral-7B achieved Ratio of Full Marks \(0.86, 0.49, 0.26, 0.13\) for \(k=2,3,4,5\) respectively [2409.19808]. Countdown RL showed out-of-distribution generalization to larger \(n\) and to unseen tree shapes, together with a stable structure-dependent hierarchy in which balanced trees are learned before left-heavy trees, and right-heavy trees remain hardest [2512.01775].

In sequential decision-making and agent orchestration, explicit structure also improves practical performance. GraSP outperforms ReAct, Reflexion, ExpeL, and flat skill baselines in every reported configuration, improving reward by up to +19 points and reducing environment steps by up to 41% [2604.17870]. SkillGraph achieves 90.6 overall success on ALFWorld and 84.4 WebShop success rate, with especially strong gains on strict-order subtasks such as Clean and Heat [2605.12039]. SkillComposer reaches 45.3% pass rate on GPT-5.2-Codex and 44.0% on Gemini-3-Pro, improving over the no-skill baseline by +23.1 pp and +18.2 pp and surpassing top-3 retrieval while using fewer prompt tokens [2606.32025]. In reward modeling, Skill-RM raises Qwen3.5-27B average benchmark performance from 83.9 for direct LLM-as-a-Judge to 86.2, and to 89.1 when sample-specific resources are mounted through the skill [2606.03980].

Robotics results show a similar divide between isolated skill competence and compositional robustness. Compose by Focus reports near-perfect single-skill performance for several baselines but a much larger gap under composition; for example, on real-world vegetable picking the Scene Graph method reaches 0.97 while Diffusion Policy scores 0.0, DP3 0.2, and \(\pi0\) 0.05 [2509.16053]. BrickCraft reports 86.25% overall success across 240 single-step assembly trials and substantial completion on an entirely unseen Castle structure in long-horizon evaluation [2605.07605]. CLASP achieves 100% (16/16) success when composing existing learned skills and 73.3% (11/15) after active acquisition and composition of grasp and insert, with the four failures concentrated in the \(180^\circ\) rotated measurement-station setting [2606.08169].

These results support a recurring pattern: explicit composition structure tends to matter most when the task requires nontrivial dependency management, conflict resolution, or generalization beyond directly observed combinations. The literature also repeatedly shows that simply exposing more skills or more context is not equivalent to structured orchestration.

## 5. Auditability, evaluation, and safety

A distinctive branch of the literature treats composition itself as something to be audited. Skill-RM reifies reward evaluation as a structured judgment
\[
z=(\mathcal E,d),\qquad \mathcal E=\{e_m\}_{m=1}^{M},\qquad e_m=(c_m,q_m,s_m),
\]
so criterion-level evidence can be inspected before the final reward readout [2606.03980]. SSL performs a related externalization for skill artifacts through a three-layer representation \(G_d=(r_{\mathrm{sch}},G_{\mathrm{str}},G_{\mathrm{log}},R_{\mathrm{cont}},R_{\mathrm{entry}})\), improving Skill Discovery MRR from 0.573 to 0.707 and Risk Assessment macro F1 from 0.744 to 0.787 [2604.24026]. Both works argue, in different settings, that source-grounded structure makes skills easier for machines to search, review, and operationalize.

Safety-oriented work shows that composition can introduce risks absent from isolated skills. SkillFuzz models a marketplace composition as an activation vector \(\mathbf{s}\in\{0,1\}^n\), extracts contracts
\[
\mathcal C(s_i)=(\mathcal P_i,\mathcal Q_i,\mathcal M_i,\mathcal V_i,\mathcal D_i,\mathcal A_i,c_i),
\]
and uses contract-guided MCTS to search for implicit intents. It reports over 1,000 distinct implicit intents, confirms 80.6% of the highest-risk flagged compositions during execution-time validation, and on the detailed strategy study finds 90 high-severity implicit intents for SkillFuzz versus 64 for Random while covering only 39.7% of pairwise interactions [2607.02345].

Multimodal evaluation studies further show that having constituent skills is not sufficient for reliable end-to-end composition. In the cross-modality benchmark, the composition gap is defined as the performance difference between cascaded and direct inference. Llama 3.2 11B improves from 55.40 to 74.46 on reasoning over rendered text and from 27.95 to 50.90 on card playing when composition is manually enforced in two stages, while Qwen2.5-VL-72B shows a much smaller gap on some tasks [2511.08113]. This supports a narrower but important point: explicit composition procedures can outperform direct inference even when the same model performs both constituent subtasks.

## 6. Limitations and open problems

The surveyed literature is also clear about its constraints. ComposeNet assumes that the correct primitive skills for the task are provided and that the form of the composition is known [1711.11289]. SUSD assumes a meaningful state factorization is given rather than learned and is sensitive to under-factorization and over-factorization [2602.01619]. GraSP notes that a DAG does not naturally represent cyclic or iterative procedures, and SkillGraph depends on a strong teacher model, known task types, and prompt-based execution rather than a learned graph encoder [2604.17870][2605.12039]. Skill-RM explicitly states that skill construction is an LLM-assisted curation process and “does not introduce an automatic skill-learning algorithm,” while also incurring higher inference cost than single-pass scalar models [2606.03980].

Robotics systems face their own boundaries. CLASP validates only pairwise compositions, depends on accurate object pose estimation, and assumes a relatively static scene [2606.08169]. Compose by Focus relies on Grounded-SAM and VLM-based relation inference, which adds computation overhead and couples performance to segmentation quality [2509.16053]. BrickCraft currently lacks strong autonomous failure recovery and remains restricted to a narrow brick family and known CAD models [2605.07605]. In multimodal reasoning, the cross-modality study shows that even simple two-stage compositions remain fragile, and composition-specific CoT or fine-tuning improves but does not fully close the gap [2511.08113].

Several open problems recur across papers. EvoTD keeps static skill and complexity taxonomies after extraction and remains tied to verifiable domains [2605.11666]. SkillFuzz depends on contract extraction quality and planner observability, and it analyzes declared intent rather than guaranteed runtime behavior [2607.02345]. SSL is presented as a practical intermediate representation rather than a finished standard or an end-to-end mechanism for skill use, and the paper explicitly leaves actual planning-time composition to future work [2604.24026]. This suggests that the next stage of the field will require tighter integration between explicit skill structure, automated skill induction, runtime verification, and adaptive composition under uncertainty, rather than simply larger skill libraries or longer prompts.

Source: https://www.emergentmind.com/topics/structured-skill-composition