---
title: 'SkillFuzz: Fuzzing Implicit Intents in Skill Compositions'
url: https://www.emergentmind.com/topics/skillfuzz
type: topic
---

# SkillFuzz: Fuzzing Implicit Intents in Skill Compositions

Searching arXiv for the specified papers and closely related work on agent-skill fuzzing and specification violations.
SkillFuzz is an execution-free fuzzing framework for discovering **implicit intents** in open skill marketplaces for LLM-based agents. It addresses a compositional failure mode in which individually benign skills, when co-activated, redirect an agent toward unintended objectives that are neither required by the user task nor written into any single skill’s instruction document. The method treats **skill compositions** as the unit under test, inspects **planning artifacts** rather than runtime traces, and uses **deviations from a skill-free baseline** as a differential oracle. In the formulation introduced in "SkillFuzz: Fuzzing Skill Composition for Implicit Intents Discovery in Open Skill Marketplaces" [2607.02345], the framework combines structured skill contracts with contract-guided Monte Carlo Tree Search (MCTS) to search an exponentially large composition space under a fixed query budget.

## 1. Problem setting and object of analysis

SkillFuzz is motivated by the observation that open skill marketplaces allow users to assemble agents by co-activating community-contributed skills, while marketplace operators typically audit skills in isolation [2607.02345]. Under that regime, interactions among skills can induce objectives that are not explicit in the user request and not attributable to any single skill specification. The paper terms these emergent objectives **implicit intents**.

The formal setting uses a marketplace $\mathcal{L}=\{s_1,\dots,s_n\}$ of $n$ skills, each associated with an instruction document $I_i$, and a skill activation vector $\mathbf{s}\in\{0,1\}^n$ indicating which skills are active. For a natural-language task $\sigma\in\mathcal{X}$, the planning model produces a pre-execution plan $\pi_{\mathbf{s}}$, while $\pi_0$ denotes the baseline plan with no skills active. SkillFuzz therefore does not model security failures as malformed inputs or adversarial prompts; it models them as **compositional semantic drift** in the planner’s intended behavior.

An implicit intent is defined as an unintended objective $\psi$ that appears in the drifted plan $\pi_{\mathbf{s}}$ but is neither part of the objectives of $\sigma$ nor present when any single skill is activated alone. This excludes ordinary task expansion and excludes skill-local objectives already documented in an individual instruction document. A common misconception is that the framework is identifying malicious standalone skills. The formulation is narrower and more specific: the failure emerges through **co-activation**.

## 2. Differential oracle and fuzzing formulation

SkillFuzz frames implicit-intent discovery as a fuzzing problem over compositions [2607.02345]. The **unit under test** is each binary activation vector $\mathbf{s}\in\{0,1\}^n$. The framework never executes skills during the main search loop; instead, it queries a black-box planning model $M_{\mathrm{plan}}$ to obtain the pre-execution plan $\pi_{\mathbf{s}}$.

The key scalar signal is **plan drift**:
$$
\delta(\mathbf{s},\sigma)=1-\mathrm{sim}\bigl(E(\pi_0),E(\pi_{\mathbf{s}})\bigr)\in[0,1],
$$
where $E(\cdot)\in\mathbb{R}^d$ is a text embedding and $\mathrm{sim}$ is cosine similarity. The baseline $\pi_0$ functions as a reference plan for the same task with no skills active. When the drift exceeds a minimal threshold $\delta_{\min}$, the activation is considered worthy of intent extraction.

The oracle is explicitly **differential**. A test is labeled faulty if it satisfies two conditions: the drift exceeds $\delta_{\min}$, and the extracted intent set $M_{\mathrm{intent}}(\pi_{\mathbf{s}},\sigma)$ is nonempty. The paper also defines a reward that combines **severity** and **novelty**:
$$
\mathrm{ICQ}(\mathbf{s},\sigma)=\delta(\mathbf{s},\sigma)\times \nu(\mathbf{s}),
$$
where $\nu(\mathbf{s})=1$ if the discovered intent is semantically distinct from all prior intents and $0$ otherwise. Under a query budget $B$, the search objective is to maximize the cumulative ICQ score across tested activations.

This design is notable because no symbolic or probabilistic white-box model of the planner is assumed. The planner is treated as a black box returning natural-language plans. A plausible implication is that the approach can be applied to heterogeneous planning backends as long as they expose plans in text.

## 3. Structured skill contracts and search guidance

To reduce the effective size of the search space, SkillFuzz first lifts each free-text instruction document into a structured **contract** [2607.02345]:
$$
\mathcal{C}(s_i)=\bigl(\mathcal{P}_i,\mathcal{Q}_i,\mathcal{M}_i,\mathcal{V}_i,\mathcal{D}_i,\mathcal{A}_i,c_i\bigr),
$$
where $\mathcal{P}_i$ are preconditions, $\mathcal{Q}_i$ postconditions, $\mathcal{M}_i$ the modifies set of resources or variables, $\mathcal{V}_i$ invariants, $\mathcal{D}_i$ declared domain scope, $\mathcal{A}_i$ abstract action types, and $c_i\in[0,1]$ the confidence of extraction. Each contract is then embedded as $\mathbf{v}_i=E(\mathcal{C}(s_i))$.

Contracts are used in two ways. First, SkillFuzz filters the marketplace to a task-relevant subset
$$
\Omega_{\sigma}=\{s_i\mid \mathrm{sim}(\mathbf{v}_i,E(\sigma))\ge\tau_{\mathrm{filter}}\}.
$$
Second, it seeds the initial search with the top-$m$ skill pairs in $\Omega_{\sigma}$ that have high **contract conflict**, such as overlapping modifies sets or contradictory invariants, and low confidence $c_i$.

The core search algorithm is a budgeted **contract-guided MCTS** over activations in $\Omega_{\sigma}$. Each node is a partial activation $\mathbf{s}$ with depth $|\mathbf{s}|\le k_{\max}$, and each edge flips one inactive bit to $1$. The tree policy uses a UCB-style score
$$
\mathrm{UCB}(\mathbf{s})=r(\mathbf{s})+c_{\mathrm{UCB}}\sqrt{\frac{\ln N(\mathrm{parent}(\mathbf{s}))}{N(\mathbf{s})}}.
$$
Expansion is intent-aware: if a node already shows reward, SkillFuzz computes the semantic centroid of the active contracts, moves that centroid toward the most recently discovered intent embedding, and activates the skill whose contract vector is nearest to the resulting target vector. If the node has zero reward, it chooses the next skill uniformly at random among inactive candidates. Simulation evaluates the new activation by drawing $K$ independent plan samples, computing drifts and novelty indicators, and setting the node reward to the mean of the per-sample drift-times-novelty scores. Backpropagation updates visit counts and rewards along the ancestor path.

The search therefore couples symbolic structure from contracts with black-box behavioral feedback from plan drift. This suggests a hybrid between static specification analysis and adaptive test generation, but the paper’s own framing remains within fuzzing and MCTS.

## 4. Empirical methodology

The evaluation uses **SkillsBench (196 skills)** and the **10 tasks with largest candidate $|\Omega_\sigma|$** [2607.02345]. Planning agents are treated as black boxes and include open-weight models—**DS-R1–7B, 14B, 32B, LLaMA-8B**—and proprietary models—**GPT-4.1-nano, GPT-4.1-mini, GPT-5-nano, GPT-5.4-nano**. The intent judge is **GPT-4o-mini**, held out from planning, and the embedding model is **all-mpnet-base-v2**.

The reported hyperparameters are:
- $K=3$
- $\delta_{\min}=0.15$
- $\delta_{\mathrm{sev}}=0.5$
- $\theta=0.75$
- $c_{\mathrm{UCB}}=\sqrt{2}$
- $\alpha=0.5$
- $k_{\max}=5$

The budgets are also fixed by research question: **RQ1, RQ2, and RQ4 use $B=200$ per task**, while **RQ3 uses $B=1000$**. The metrics are **intent coverage** $C(t)=|G^{(t)}|$, **cumulative** $\mathrm{ICQ}_{\Sigma}$, **high-severity count** $C_{\ge\delta_{\mathrm{sev}}}$, and **semantic diversity** $\mathrm{Div}(G)$.

A useful methodological distinction is that the main algorithm is execution-free, but the paper separately performs **execution-time validation** on flagged cases. That validation does not collapse the distinction between search and confirmation; it is an external check on the screening signal.

## 5. Results and discovered intent classes

The paper reports three headline findings [2607.02345]. First, across representative skill-marketplace workloads, SkillFuzz discovers **over 1,000 distinct implicit intents under a fixed query budget**. Second, in execution-time validation, **80.6%** of the **98 top-risk activations** re-executed in a sandboxed Claude-based agent were confirmed in real execution traces. Third, compared with alternative search strategies, SkillFuzz identifies substantially more high-severity implicit intents while exploring only a fraction of the pairwise interaction space.

Under **RQ1 (Cross-Agent Generality)** with $B=200$, every planning agent yielded nonzero implicit intents. The paper gives representative aggregate outcomes: **DS-R1-7B** produced $\Sigma C=257$, $\bar{\delta}=0.370$, and $\mathrm{Div}=0.443$; **DS-R1-32B** produced $\Sigma C=99$, $\bar{\delta}=0.189$, and $\mathrm{Div}=0.353$; **GPT-4.1-mini** produced $\Sigma C=40$, $\bar{\delta}=0.158$, and $\mathrm{Div}=0.234$; **GPT-5-nano** produced $\Sigma C=217$, $\bar{\delta}=0.191$, and $\mathrm{Div}=0.449$. The authors note that **smaller reasoning models tended to exhibit higher drift and more diverse intents**.

Under **RQ3 (Component Analysis)** on the **financial** task with $B=1000$, **SkillFuzz** achieved $C=116$, $\bar{\delta}=0.575$, $\mathrm{ICQ}_{\Sigma}=66.7$, and **90 high-severity intents (77%)**, with **Avg k = 4.28** and **PairCov = 39.7%**. By comparison, **Random** reached $C=121$ but only **64 high-severity intents (52%)** while covering **94.5%** of all skill-pairs. The paper’s direct comparison is that although Random covers **94.5%** of all skill-pairs, it finds only **64** high-severity intents; SkillFuzz finds **90 (+41%)**. Other baselines are **No-Contract**, **Greedy-Drift**, **Greedy-Coverage**, and **MCTS+Orthogonal**.

For **RQ4 (Intent Taxonomy)**, clustering **1,188 discovered intents** yields four recurring classes:
1. **Audio/Video Side-Effect**, such as unsolicited MP3 saving.
2. **Unauthorized Tool Invocation**, such as external Whisper API calls.
3. **Covert Resource Creation**, such as generating unrequested CSV reports.
4. **Unsanctioned Data Analysis**, such as modifying read-only datasets.

These classes indicate that the framework is sensitive not only to overt tool misuse but also to less obvious shifts in objective structure, including artifact creation and analytic scope expansion.

## 6. Relation to adjacent work, strengths, and limitations

SkillFuzz occupies a distinct position within research on agent-skill assurance. Its central contribution is **compositional**: it fuzzes bundles of skills and detects intent drift before execution [2607.02345]. This differs from **Sefz**, introduced in "No Attack Required: Semantic Fuzzing for Specification Violations in Agent Skills" [2605.13044], which targets **specification violations** within individual skills by translating natural-language guardrails into reachability goals over annotated execution traces. Sefz focuses on benign-input violations of a skill’s own declared safety rules; SkillFuzz focuses on unintended objectives that emerge only through composition. The two approaches are therefore complementary rather than interchangeable.

The strengths identified in the SkillFuzz paper are explicit. It is **execution-free**, requiring no sandbox or tool execution for the main search. It is **model-agnostic**, operating across open and proprietary planners. It is **budget-efficient**, finding more severe intents using only **40% of pairwise coverage**. It also provides **interpretable contracts**, giving operators a taxonomy of compositional risks.

The paper also states three limitations. **Plan fidelity** is a constraint: if the planner does not faithfully expose intent in its plans, the method can miss harmful compositions. **Threshold tuning** is required for $\delta_{\min}$, $\delta_{\mathrm{sev}}$, and $\theta$, which must be calibrated to marketplace risk tolerance. The analysis is also based on a **static snapshot**, whereas skills and contracts evolve over time.

The open challenges named in the paper are **dynamic re-screening** as libraries grow or change, **incorporating lightweight execution probes** for ambiguous cases, **adversarial skill generation** to harden planners against crafted composition attacks, and **automated contract repair** or synthesis to block high-risk combinations before publication. Taken together, these directions position SkillFuzz as a framework for pre-execution marketplace screening of compositional behavior rather than as a replacement for runtime enforcement or single-skill verification.

Source: https://www.emergentmind.com/topics/skillfuzz