Skill Composition Risk: Metrics & Mitigations

• Skill Composition Risk (SCR) is the measure of risk arising from combining individual skills, where structure- and path-dependent failure modes emerge undetected by isolated evaluations.
• It spans diverse domains including reinforcement learning, agent orchestration, and robotics by employing compositional objects such as trees, graphs, and adapters.
• SCR research develops structure-aware metrics and diagnostic frameworks that guide mitigation strategies and improve safety in multi-skill systems.

Skill Composition Risk (SCR) denotes the risk that emerges specifically from combining skills rather than evaluating each skill in isolation. Recent work uses the term across several technical settings: compositional reasoning in reinforcement-learning post-training, agent skill orchestration, security analysis of activated multi-skill paths, execution-free discovery of implicit intents in open marketplaces, dependency-bearing skill supply chains, robot skill fusion, hybrid workforce design, and LoRA-based model merging. Although the formal objects differ—expression trees, typed skill graphs, activated paths, dependency graphs, Gaussian trajectory compositions, or merged low-rank updates—the recurring claim is that composition introduces structure-dependent, path-dependent, or dependency-dependent failure modes that are invisible to isolated evaluation and often poorly captured by aggregate metrics such as pass@k or per-skill vetting (Park et al., 1 Dec 2025, Liang et al., 27 Apr 2026, Xie et al., 13 Jun 2026, Hu et al., 2 Jul 2026).

1. Domain-specific meanings of SCR

The literature does not present a single universal SCR formalism. Instead, it defines closely related risk notions over different compositional objects. Several papers do not use the term explicitly, but derive a composition-specific risk quantity from their own framework.

Domain	Compositional object	SCR notion
RL reasoning	Canonical expression trees	$\mathrm{SCR}_t(C)=1-P_t(\text{success}\mid C)$
Agent orchestration	Composed skill graphs	Incremental risk beyond isolated skills
Path-aware agent security	Activated skill path $\pi$	$r_\pi(h)=a_\pi(h)q_\pi(h)$
Marketplace fuzzing	Skill activation vector $s$	Plan drift and implicit-intent discovery
Robot skill composition	TP-KMP fusion	Risk from incompatibility, uncertainty, and unmet preconditions
Hybrid workforce	Skill-critical onsite subset	Expected infections among skill-critical members
LoRA merging	Merged skill adapters	Risk of negative transfer and ineffective composition

In the Countdown study, skills are subtrees of canonical arithmetic-expression trees, and SCR is defined directly as the complement of empirical success probability for a tree or tree class over training time (Park et al., 1 Dec 2025). In SSL-based agent analysis, SCR is the incremental risk introduced when multiple skills are combined and orchestrated beyond the sum of risks of each skill in isolation (Liang et al., 27 Apr 2026). In SCR-Bench, the unit of analysis is the activated composition path through shared execution context, and risk lives on edges that carry outputs, endorsements, or authorization-like cues into later skill invocations (Xie et al., 13 Jun 2026). SkillReact defines compositional risk over individually safe skill pairs whose capability union satisfies a forbidden conjunction (Wang et al., 30 May 2026). SkillFuzz instead treats compositions as fuzzing inputs and identifies implicit intents that arise only under co-activation (Hu et al., 2 Jul 2026).

This suggests that SCR is best understood as a family of structure-aware risk measures rather than a single metric. The common invariant is counterfactual: a composition is risky when its failure probability, harmful state probability, or unintended objective cannot be recovered from the properties of any constituent skill in isolation.

2. Formal representations and risk functions

In the Countdown formulation, a solution is parsed as a canonical binary expression tree $T$ whose leaves are input numbers and whose internal nodes are drawn from $\{+, -, \times, \div\}$. A root-level shape signature $[x]\circ[y]$ records the leaf counts in the left and right subtrees, composition depth $d(T)$ is the maximum operator depth, and two scalar structural summaries are defined:

$$b(T)=\frac{|x-y|}{x+y}\in[0,1], \qquad r(T)=\frac{y-x}{x+y}\in[-1,1].$$

Here $b(T)$ measures workload asymmetry and $\pi$0 measures directioned heaviness. For a tree class $\pi$1 at training step $\pi$2,

$\pi$3

and

$\pi$4

For an individual tree, the paper further gives an independence approximation over subskills $\pi$5 with failure probabilities $\pi$6:

$\pi$7

The paper’s central caveat is that right-heavy trees violate this approximation because root choice and later subtree construction are coupled by a “lookahead bottleneck” (Park et al., 1 Dec 2025).

Agent-skill security work typically uses graphs rather than trees. SCR-Bench models a context-dependent composition graph $\pi$8, with edges

$\pi$9

where $r_\pi(h)=a_\pi(h)q_\pi(h)$0 is an output or state produced by skill $r_\pi(h)=a_\pi(h)q_\pi(h)$1 and $r_\pi(h)=a_\pi(h)q_\pi(h)$2 is consumed by a later skill or planner. A path

$r_\pi(h)=a_\pi(h)q_\pi(h)$3

has risk

$r_\pi(h)=a_\pi(h)q_\pi(h)$4

with $r_\pi(h)=a_\pi(h)q_\pi(h)$5 the activation probability and $r_\pi(h)=a_\pi(h)q_\pi(h)$6 the conditional probability of reaching a harmful state once activated. The appendix gives an ecosystem-level approximation

$r_\pi(h)=a_\pi(h)q_\pi(h)$7

showing that small average path risk can compound when many feasible paths exist (Xie et al., 13 Jun 2026).

SkillReact formalizes capability-level compositional risk over a fixed taxonomy $r_\pi(h)=a_\pi(h)q_\pi(h)$8 and a library of forbidden conjunctions. A pair is a structural candidate when each member is individually safe but the union of their capability bitsets satisfies at least one forbidden pattern. This makes SCR explicitly set-level and policy-relative rather than exploit-realization-relative (Wang et al., 30 May 2026).

SkillFuzz uses a differential plan-space formalization. For task $r_\pi(h)=a_\pi(h)q_\pi(h)$9, baseline plan $s$0, and composed plan $s$1 under activation vector $s$2, plan drift is

$s$3

Intent Coverage Quality is

$s$4

where $s$5 is a novelty indicator. An implicit intent is an unintended objective that appears in $s$6 but is not required by the task and not prescribed by any individual activated skill (Hu et al., 2 Jul 2026).

Other domains instantiate the same pattern with different state spaces. SSL represents skills as a typed, source-grounded JSON graph with scheduling, structural, and logical layers, enabling risk checks over interfaces, scene transitions, and atomic actions such as READ, WRITE, CALL_TOOL, REQUEST, or TRANSFER (Liang et al., 27 Apr 2026). SkillOps models each skill as a contract $s$7 and propagates local risk through dependency edges with

$s$8

so downstream skills inherit upstream composition risk (Pu et al., 13 May 2026). CLASP fuses Gaussian trajectory distributions from local TP-KMPs and rejects composition when the variance-dominance compatibility constraint fails (Knauer et al., 6 Jun 2026). In hybrid workforce optimization, SCR is derived as expected infections among the minimal skill-cover subset of onsite employees, and in LoRA merging it is operationalized as failure to preserve or combine constituent skills under merge-induced interference (Hung et al., 28 Feb 2026, Prabhakar et al., 2024).

3. Measurement frameworks and diagnostics

A distinguishing feature of SCR research is the replacement of coarse aggregate evaluation with composition-aware diagnostics. In Countdown, the diagnostic pipeline parses each final answer into a canonical tree, aggregates per-pattern precision by root operator and root shape, and tracks coverage, per-pattern precision, and high-precision coverage over training checkpoints. The training setup is Qwen2.5 1.5B, 3B, and 7B models, RL post-training with GRPO on $s$9 for one epoch, a $T$0 format reward plus $T$1 valid-and-correct reward, checkpointing every $T$2 updates, and evaluation with $T$3 samples per held-out question at temperature $T$4 (Park et al., 1 Dec 2025).

SkillReact separates static overapproximation, human calibration, and dynamic realization. Its deterministic benchmark computes capability profiles and pairwise structural candidates over a frozen registry snapshot; its two-rater LLM-assisted human-adjudication pipeline calibrates flagged pair-pattern memberships; and its action-based exploitability harness measures when candidates become model-issued tool calls under permissive sandbox conditions (Wang et al., 30 May 2026). SCR-Bench is similarly path-aware but measures realized downstream state changes in sandboxed environments. It defines three sub-benchmarks: SCR-CapFlow for capability-flow composition, SCR-TrustLift for trust-transfer composition, and SCR-AuthBlur for authorization-confusion composition. Harm is counted only when a concrete bad event occurs, not when a model merely describes one (Xie et al., 13 Jun 2026).

SkillFuzz moves admission-time testing to the planning surface. It extracts structured skill contracts

$T$5

filters task-relevant candidates, seeds conflict-prone compositions, and uses contract-guided Monte Carlo Tree Search with UCB selection, bit-flip mutations, and a differential oracle on planning artifacts. The reported hyperparameters are $T$6, $T$7, $T$8, $T$9, $\{+, -, \times, \div\}$0, $\{+, -, \times, \div\}$1, and $\{+, -, \times, \div\}$2 (Hu et al., 2 Jul 2026).

Static representation work emphasizes machine-readable structure as a prerequisite for measuring SCR. SSL converts SKILL.md artifacts into a typed graph via a four-pass constrained NL2JSON pipeline and evaluates the representation on Skill Discovery and Risk Assessment (Liang et al., 27 Apr 2026). SkillOps uses a Hierarchical Skill Ecosystem Graph with dependency, compatibility, redundancy, and alternative edges, and summarizes composition-related health via utility $\{+, -, \times, \div\}$3, redundancy $\{+, -, \times, \div\}$4, compatibility $\{+, -, \times, \div\}$5, failure-risk $\{+, -, \times, \div\}$6, and validation-gap $\{+, -, \times, \div\}$7 (Pu et al., 13 May 2026). SkillDepAnalyzer reconstructs Agent Skill Supply Chains (ASSCs) across skill, package, and service dependencies and reports amplification, hidden inventory, concentration, clustering, and dependency-only exposures (Jia et al., 1 Jul 2026).

4. Empirical regularities

The Countdown study shows that RL-only post-training induces strong out-of-distribution generalization in length while leaving measurable composition-specific risk. Models trained only on $\{+, -, \times, \div\}$8 and $\{+, -, \times, \div\}$9 generalize to $[x]\circ[y]$0 with pass@32 of approximately $[x]\circ[y]$1, $[x]\circ[y]$2, and $[x]\circ[y]$3 for Qwen2.5 1.5B, 3B, and 7B respectively, and to $[x]\circ[y]$4 with pass@32 of approximately $[x]\circ[y]$5–$[x]\circ[y]$6. Yet the per-pattern diagnostic reveals a stable hierarchy: shallow balanced trees are learned earliest, deep unbalanced trees lag, and right-heavy structures remain fragile even at the same depth as left-heavy counterparts. For Qwen2.5-1.5B at $[x]\circ[y]$7, the paper reports division-root precision $[x]\circ[y]$8 for balanced $[x]\circ[y]$9, $d(T)$0 for left-heavy $d(T)$1, and $d(T)$2 for right-heavy $d(T)$3, corresponding to SCR values of approximately $d(T)$4, $d(T)$5, and $d(T)$6. Aggregated by root operator on $d(T)$7, precision is $d(T)$8 for $d(T)$9, $$b(T)=\frac{|x-y|}{x+y}\in[0,1], \qquad r(T)=\frac{y-x}{x+y}\in[-1,1].$$0 for $$b(T)=\frac{|x-y|}{x+y}\in[0,1], \qquad r(T)=\frac{y-x}{x+y}\in[-1,1].$$1, $$b(T)=\frac{|x-y|}{x+y}\in[0,1], \qquad r(T)=\frac{y-x}{x+y}\in[-1,1].$$2 for $$b(T)=\frac{|x-y|}{x+y}\in[0,1], \qquad r(T)=\frac{y-x}{x+y}\in[-1,1].$$3, and $$b(T)=\frac{|x-y|}{x+y}\in[0,1], \qquad r(T)=\frac{y-x}{x+y}\in[-1,1].$$4 for $$b(T)=\frac{|x-y|}{x+y}\in[0,1], \qquad r(T)=\frac{y-x}{x+y}\in[-1,1].$$5 (Park et al., 1 Dec 2025).

SkillReact provides a calibrated estimate of structural compositional risk in an open skill registry. From 1,520 ClawHub skills, 869 are individually flagged and 651 pass individual inspection, yielding 211,575 unordered pairs of individually safe skills. The static benchmark flags 47,075 candidate pairs, so the Compositional Candidate Rate is $$b(T)=\frac{|x-y|}{x+y}\in[0,1], \qquad r(T)=\frac{y-x}{x+y}\in[-1,1].$$6 with bootstrap $$b(T)=\frac{|x-y|}{x+y}\in[0,1], \qquad r(T)=\frac{y-x}{x+y}\in[-1,1].$$7 CI $$b(T)=\frac{|x-y|}{x+y}\in[0,1], \qquad r(T)=\frac{y-x}{x+y}\in[-1,1].$$8. At the membership level, the scanner finds 78,878 flagged pair-pattern hits. Human calibration yields a population-weighted pair-pattern validity of $$b(T)=\frac{|x-y|}{x+y}\in[0,1], \qquad r(T)=\frac{y-x}{x+y}\in[-1,1].$$9 with $b(T)$0 CI $b(T)$1, implying approximately 14,356 valid risk memberships. Under the co-validation assumption, the projected pair-level rate is approximately $b(T)$2; the assumption-free ceiling derived from membership counts is $b(T)$3 (Wang et al., 30 May 2026).

SCR-Bench shows that activated paths expose risks largely absent under isolated evaluation. In SCR-CapFlow, Control and A-Only yield $b(T)$4 ASR across backends and B-Only averages $b(T)$5, whereas A+B Neutral averages $b(T)$6 and A+B Explicit averages $b(T)$7. In SCR-TrustLift, average ASR rises from $b(T)$8 in the control condition to $b(T)$9 in the endorsed condition, with lifts of $\pi$00 percentage points for Claude Opus 4.5 and MiniMax-M2.7, $\pi$01 for GPT-5.4, and $\pi$02 for Gemini 3.1 Pro Preview. In SCR-AuthBlur, average risky approval increases from $\pi$03 at $\pi$04 to $\pi$05 at $\pi$06 and $\pi$07 at $\pi$08; the abstract reports a $\pi$09 relative increase at $\pi$10 compared with the $\pi$11 isolated baseline (Xie et al., 13 Jun 2026).

SkillFuzz reports more than 1,000 distinct implicit intents under a fixed query budget and validates the top 98 highest-risk compositions end to end, with an overall confirmation rate of $\pi$12. On a representative task with budget $\pi$13, SkillFuzz discovers 90 high-severity intents with $\pi$14, versus 64 for random sampling, while exploring $\pi$15 of all pairs rather than Random’s $\pi$16. Severe drift is rare for singleton activations but rises monotonically with co-activation depth, reaching $\pi$17 severe plans at $\pi$18, which the paper describes as a fourteen-fold increase relative to singletons (Hu et al., 2 Jul 2026).

Representation-centric and ecosystem-scale studies report additional regularities. SSL improves Skill Discovery MRR from $\pi$19 to $\pi$20 and Risk Assessment macro-F1 from $\pi$21 to $\pi$22 when combined with SKILL.md (Liang et al., 27 Apr 2026). SkillOps reaches $\pi$23 percent task success as a standalone agent on ALFWorld, outperforming the strongest baseline by $\pi$24 percentage points, and improves retrieval-heavy baselines by $\pi$25 to $\pi$26 percentage points as a plug-in layer (Pu et al., 13 May 2026). SkillDepAnalyzer recovers ASSCs over 1,434,046 retrieved GitHub-backed skills from a 1,640,440-skill snapshot and finds channel coverage of $\pi$27 for skill dependencies, $\pi$28 for packages, $\pi$29 for services, and $\pi$30 for any dependency channel, with Gini coefficients of $\pi$31 for skill reuse and $\pi$32 for package reuse (Jia et al., 1 Jul 2026).

Outside agent ecosystems, CLASP reports $\pi$33 success on 16 composed pick-and-place behaviors, $\pi$34 correct capability-gap detection, and $\pi$35 success on composed grasp-and-insert after acquiring an insert skill, with failures concentrated at $\pi$36 rotations relative to training (Knauer et al., 6 Jun 2026). In LoRA-based model merging, CAT reaches $\pi$37 execution accuracy on GSM-Hard for math-plus-code composition, outputs code at least $\pi$38 of the time, and is reported to outperform other model-merging methods by an average of $\pi$39 and data-mixing by an average of $\pi$40 on math-word problems (Prabhakar et al., 2024).

5. Mechanisms and failure modes

The empirical literature converges on several recurrent mechanisms. In reasoning tasks, structural asymmetry is itself a risk source: higher depth, larger balance metric, and positive right-heaviness increase SCR, with right-heavy trees showing excess risk beyond an independence approximation because committing to a root operator requires anticipating a complex deferred subtree. The paper identifies this as a planning or commitment bottleneck rather than a simple length effect (Park et al., 1 Dec 2025).

Agent-skill security papers replace structural asymmetry with inter-skill state transfer. SSL enumerates composition-time hazards as conflicting preconditions, unsafe tool invocation sequences, privilege aggregation, unsafe data flow, compounded non-idempotent side effects, deadlocks, unsafe loops, and brittle error handling. The logical layer makes patterns such as USER_DATA $\pi$41 TRANSFER $\pi$42 NETWORK or READ(CREDENTIALS) $\pi$43 CALL_TOOL directly machine-checkable (Liang et al., 27 Apr 2026). SCR-Bench sharpens this into three path mechanisms: capability flow, trust transfer, and authorization confusion. These mechanisms differ in the intermediate artifact that crosses the edge—execution target, endorsement, or approval-like cue—but all exploit the shared execution context rather than any overtly harmful isolated skill (Xie et al., 13 Jun 2026).

Marketplace and supply-chain studies add two further mechanisms. First, belief formation is non-decomposable: SkillFuzz argues that the joint plan under co-activation cannot be reconstructed from singleton plans, which is why implicit intents such as unrequested artifact creation, unauthorized tool invocation, unsanctioned data manipulation, and unauthorized output-format substitution appear only under composition (Hu et al., 2 Jul 2026). Second, skills are dependency-bearing artifacts with hidden transitive surface area. ASSC analysis reports recursive skill reuse, hidden package inventory, name collisions, sparse version and license declaration, concentrated reuse, cycles, and workflow-centric strongly connected components. These phenomena make security and governance cluster-level rather than artifact-local (Jia et al., 1 Jul 2026).

Maintenance-oriented work identifies additional failure channels that are not primarily adversarial. SkillOps defines interface mismatch, missing validators, high empirical failure rates, redundancy-induced wrong selection, and upstream-downstream propagated risk as the main composition hazards in skill libraries. Its ablations show that removing add_adapter collapses success to 13.2–13.9%, removing add_validator drops success to 38.0–38.8%, and removing repair reduces success to about 56%, which directly ties composition reliability to typed adapters and validation barriers (Pu et al., 13 May 2026). Proteus adds an adaptive-adversary perspective: residual adaptive leakage remains substantial after audit, and an ablation forcing single-skill rather than chain-based attacks drops stars from 80% to 0% on a hard AIG cell, indicating that multi-skill chains are load-bearing for bypass and harm (Zhou, 12 May 2026).

In robotic skill composition, failure modes are physical rather than informational: incompatibility of local skills under the covariance-dominance constraint, large fused covariance near contact, unmet preconditions, weak parameter binding, perception errors in task frames, and genuine capability gaps (Knauer et al., 6 Jun 2026). In hybrid workforce configuration, SCR is driven by the need to place skill-critical employees onsite within a contact graph, so replacing a skill-critical member with an alternative who preserves coverage but has lower infection probability reduces composition-specific risk (Hung et al., 28 Feb 2026). In LoRA merging, the analogous hazards are negative transfer, domain conflicts, capacity bottlenecks, catastrophic forgetting under data mixing, and cross-terms introduced by linear parameter merging (Prabhakar et al., 2024).

6. Mitigation, governance, and open problems

Mitigation strategies are consistently structure-aware. In Countdown, the recommended interventions are curriculum emphasis on structurally hard instances—especially right-heavy and deep unbalanced trees with $\pi$44 or $\pi$45 roots—balanced data generation over canonical patterns, tracking $\pi$46 alongside coverage and high-precision coverage, monitoring the gap between pass@k and average accuracy, and avoiding reward settings that encourage formatting-only shortcuts. The paper also notes a trade-off between GRPO and PPO, and reports that standard deviation normalization in group advantage increased response length via negative gradients and hurt final performance (Park et al., 1 Dec 2025).

Security-oriented work shifts defenses from isolated skills to installed sets and activated paths. SkillReact recommends install-time compositional checks, capability isolation and scoping, runtime allowlists, scoped credentials, per-call confirmations, and registry-level publication of evidence-backed pair indicators (Wang et al., 30 May 2026). SCR-Bench argues for path-aware vetting, capability scoping, provenance-sensitive gating, explicit decision barriers, memory and side-effect isolation, and pre-execution firewalls that validate tool calls against path-aware policies (Xie et al., 13 Jun 2026). SkillFuzz adds execution-free admission testing, contract linting, read-only defaults, explicit opt-in for write or network capability, runtime monitors for outbound calls and file writes, and periodic re-screening as skills evolve (Hu et al., 2 Jul 2026). Proteus recommends adversarial-in-the-loop vetting, multi-skill differential testing, randomized audits and feedback obfuscation, formal conformance checks on documentation-code alignment, semantic sandboxing, cross-skill policy guards, provenance controls, and layered auditors robust to feedback-driven evasion (Zhou, 12 May 2026).

Representation and maintenance papers emphasize machine-readable structure and library-time repair. SSL recommends storing SKILL.md alongside SSL JSON, enforcing entry conditions and expected input-output contracts at composition time, tracing data and permission flow across composite graphs, defining policy gates for egress, credentials, destructive operations, and loop bounds, and logging evidence-rich remediation workflows (Liang et al., 27 Apr 2026). SkillOps operationalizes mitigation through merge, repair, retire, add_validator, and add_adapter, with maintenance triggered by low utility, high redundancy, low compatibility, high failure-risk, high validation-gap, or high propagated risk (Pu et al., 13 May 2026). ASSC analysis recommends typed dependency manifests, first-class dependency-cluster management, risk-warning audit commands, and lockfile-like records with exact versions, repositories, paths, optionality, and integrity hashes (Jia et al., 1 Jul 2026).

Broader domains present analogous controls. CLASP rejects incompatible compositions by a hard feasibility gate, uses covariance-weighted fusion only when local skills dominate disjoint temporal regions, and triggers active learning when no single skill or safe composition exists (Knauer et al., 6 Jun 2026). The RSHWC framework reduces SCR by risk-aware workforce construction, skill-preserving refinement, and risk-reducing member replacement under a fixed risk budget (Hung et al., 28 Feb 2026). LoRA Soups recommends freezing constituent skill adapters, learning layer-wise scalar merge weights on a tiny balanced dataset, and preferring data mixing when the number of skills increases to three or more, since the paper’s three-format prompt-robustness study finds data mixing superior in that setting (Prabhakar et al., 2024).

Open problems are similarly convergent. Many studies are pairwise or task-specific; several explicitly note that static models overapproximate execution; service transitives, dynamic runtime context, and larger $\pi$47-way compositions remain insufficiently modeled; and dependence typically makes simple independence baselines understate true composition risk. This suggests that future SCR research will continue to move toward typed manifests, richer contracts, path-aware or graph-aware evaluation, human-calibrated or execution-backed validation, and training or governance interventions that target structural bottlenecks directly rather than relying on artifact-local scores alone.