SkillFuzz: Fuzzing Implicit Intents in Skill Compositions

• SkillFuzz is an execution-free fuzzing framework that detects implicit intents emerging from co-activated skills in LLM agents.
• It combines structured skill contracts with contract-guided Monte Carlo Tree Search to efficiently navigate large, compositional search spaces.
• Empirical results demonstrate the method uncovers over 1,000 distinct implicit intents with high severity, guiding risk mitigation in open skill marketplaces.

Searching arXiv for the specified papers and closely related work on agent-skill fuzzing and specification violations. SkillFuzz is an execution-free fuzzing framework for discovering implicit intents in open skill marketplaces for LLM-based agents. It addresses a compositional failure mode in which individually benign skills, when co-activated, redirect an agent toward unintended objectives that are neither required by the user task nor written into any single skill’s instruction document. The method treats skill compositions as the unit under test, inspects planning artifacts rather than runtime traces, and uses deviations from a skill-free baseline as a differential oracle. In the formulation introduced in "SkillFuzz: Fuzzing Skill Composition for Implicit Intents Discovery in Open Skill Marketplaces" (Hu et al., 2 Jul 2026), the framework combines structured skill contracts with contract-guided Monte Carlo Tree Search (MCTS) to search an exponentially large composition space under a fixed query budget.

1. Problem setting and object of analysis

SkillFuzz is motivated by the observation that open skill marketplaces allow users to assemble agents by co-activating community-contributed skills, while marketplace operators typically audit skills in isolation (Hu et al., 2 Jul 2026). Under that regime, interactions among skills can induce objectives that are not explicit in the user request and not attributable to any single skill specification. The paper terms these emergent objectives implicit intents.

The formal setting uses a marketplace $\mathcal{L}=\{s_1,\dots,s_n\}$ of $n$ skills, each associated with an instruction document $I_i$, and a skill activation vector $\mathbf{s}\in\{0,1\}^n$ indicating which skills are active. For a natural-language task $\sigma\in\mathcal{X}$, the planning model produces a pre-execution plan $\pi_{\mathbf{s}}$, while $\pi_0$ denotes the baseline plan with no skills active. SkillFuzz therefore does not model security failures as malformed inputs or adversarial prompts; it models them as compositional semantic drift in the planner’s intended behavior.

An implicit intent is defined as an unintended objective $\psi$ that appears in the drifted plan $\pi_{\mathbf{s}}$ but is neither part of the objectives of $\sigma$ nor present when any single skill is activated alone. This excludes ordinary task expansion and excludes skill-local objectives already documented in an individual instruction document. A common misconception is that the framework is identifying malicious standalone skills. The formulation is narrower and more specific: the failure emerges through co-activation.

2. Differential oracle and fuzzing formulation

SkillFuzz frames implicit-intent discovery as a fuzzing problem over compositions (Hu et al., 2 Jul 2026). The unit under test is each binary activation vector $n$0. The framework never executes skills during the main search loop; instead, it queries a black-box planning model $n$1 to obtain the pre-execution plan $n$2.

The key scalar signal is plan drift:

$n$3

where $n$4 is a text embedding and $n$5 is cosine similarity. The baseline $n$6 functions as a reference plan for the same task with no skills active. When the drift exceeds a minimal threshold $n$7, the activation is considered worthy of intent extraction.

The oracle is explicitly differential. A test is labeled faulty if it satisfies two conditions: the drift exceeds $n$8, and the extracted intent set $n$9 is nonempty. The paper also defines a reward that combines severity and novelty:

$I_i$0

where $I_i$1 if the discovered intent is semantically distinct from all prior intents and $I_i$2 otherwise. Under a query budget $I_i$3, the search objective is to maximize the cumulative ICQ score across tested activations.

This design is notable because no symbolic or probabilistic white-box model of the planner is assumed. The planner is treated as a black box returning natural-language plans. A plausible implication is that the approach can be applied to heterogeneous planning backends as long as they expose plans in text.

3. Structured skill contracts and search guidance

To reduce the effective size of the search space, SkillFuzz first lifts each free-text instruction document into a structured contract (Hu et al., 2 Jul 2026):

$I_i$4

where $I_i$5 are preconditions, $I_i$6 postconditions, $I_i$7 the modifies set of resources or variables, $I_i$8 invariants, $I_i$9 declared domain scope, $\mathbf{s}\in\{0,1\}^n$0 abstract action types, and $\mathbf{s}\in\{0,1\}^n$1 the confidence of extraction. Each contract is then embedded as $\mathbf{s}\in\{0,1\}^n$2.

Contracts are used in two ways. First, SkillFuzz filters the marketplace to a task-relevant subset

$\mathbf{s}\in\{0,1\}^n$3

Second, it seeds the initial search with the top-$\mathbf{s}\in\{0,1\}^n$4 skill pairs in $\mathbf{s}\in\{0,1\}^n$5 that have high contract conflict, such as overlapping modifies sets or contradictory invariants, and low confidence $\mathbf{s}\in\{0,1\}^n$6.

The core search algorithm is a budgeted contract-guided MCTS over activations in $\mathbf{s}\in\{0,1\}^n$7. Each node is a partial activation $\mathbf{s}\in\{0,1\}^n$8 with depth $\mathbf{s}\in\{0,1\}^n$9, and each edge flips one inactive bit to $\sigma\in\mathcal{X}$0. The tree policy uses a UCB-style score

$\sigma\in\mathcal{X}$1

Expansion is intent-aware: if a node already shows reward, SkillFuzz computes the semantic centroid of the active contracts, moves that centroid toward the most recently discovered intent embedding, and activates the skill whose contract vector is nearest to the resulting target vector. If the node has zero reward, it chooses the next skill uniformly at random among inactive candidates. Simulation evaluates the new activation by drawing $\sigma\in\mathcal{X}$2 independent plan samples, computing drifts and novelty indicators, and setting the node reward to the mean of the per-sample drift-times-novelty scores. Backpropagation updates visit counts and rewards along the ancestor path.

The search therefore couples symbolic structure from contracts with black-box behavioral feedback from plan drift. This suggests a hybrid between static specification analysis and adaptive test generation, but the paper’s own framing remains within fuzzing and MCTS.

4. Empirical methodology

The evaluation uses SkillsBench (196 skills) and the 10 tasks with largest candidate $\sigma\in\mathcal{X}$3 (Hu et al., 2 Jul 2026). Planning agents are treated as black boxes and include open-weight models—DS-R1–7B, 14B, 32B, LLaMA-8B—and proprietary models—GPT-4.1-nano, GPT-4.1-mini, GPT-5-nano, GPT-5.4-nano. The intent judge is GPT-4o-mini, held out from planning, and the embedding model is all-mpnet-base-v2.

The reported hyperparameters are:

• $\sigma\in\mathcal{X}$4
• $\sigma\in\mathcal{X}$5
• $\sigma\in\mathcal{X}$6
• $\sigma\in\mathcal{X}$7
• $\sigma\in\mathcal{X}$8
• $\sigma\in\mathcal{X}$9
• $\pi_{\mathbf{s}}$0

The budgets are also fixed by research question: RQ1, RQ2, and RQ4 use $\pi_{\mathbf{s}}$1 per task, while RQ3 uses $\pi_{\mathbf{s}}$2. The metrics are intent coverage $\pi_{\mathbf{s}}$3, cumulative $\pi_{\mathbf{s}}$4, high-severity count $\pi_{\mathbf{s}}$5, and semantic diversity $\pi_{\mathbf{s}}$6.

A useful methodological distinction is that the main algorithm is execution-free, but the paper separately performs execution-time validation on flagged cases. That validation does not collapse the distinction between search and confirmation; it is an external check on the screening signal.

5. Results and discovered intent classes

The paper reports three headline findings (Hu et al., 2 Jul 2026). First, across representative skill-marketplace workloads, SkillFuzz discovers over 1,000 distinct implicit intents under a fixed query budget. Second, in execution-time validation, 80.6% of the 98 top-risk activations re-executed in a sandboxed Claude-based agent were confirmed in real execution traces. Third, compared with alternative search strategies, SkillFuzz identifies substantially more high-severity implicit intents while exploring only a fraction of the pairwise interaction space.

Under RQ1 (Cross-Agent Generality) with $\pi_{\mathbf{s}}$7, every planning agent yielded nonzero implicit intents. The paper gives representative aggregate outcomes: DS-R1-7B produced $\pi_{\mathbf{s}}$8, $\pi_{\mathbf{s}}$9, and $\pi_0$0; DS-R1-32B produced $\pi_0$1, $\pi_0$2, and $\pi_0$3; GPT-4.1-mini produced $\pi_0$4, $\pi_0$5, and $\pi_0$6; GPT-5-nano produced $\pi_0$7, $\pi_0$8, and $\pi_0$9. The authors note that smaller reasoning models tended to exhibit higher drift and more diverse intents.

Under RQ3 (Component Analysis) on the financial task with $\psi$0, SkillFuzz achieved $\psi$1, $\psi$2, $\psi$3, and 90 high-severity intents (77%), with Avg k = 4.28 and PairCov = 39.7%. By comparison, Random reached $\psi$4 but only 64 high-severity intents (52%) while covering 94.5% of all skill-pairs. The paper’s direct comparison is that although Random covers 94.5% of all skill-pairs, it finds only 64 high-severity intents; SkillFuzz finds 90 (+41%). Other baselines are No-Contract, Greedy-Drift, Greedy-Coverage, and MCTS+Orthogonal.

For RQ4 (Intent Taxonomy), clustering 1,188 discovered intents yields four recurring classes:

1. Audio/Video Side-Effect, such as unsolicited MP3 saving.
2. Unauthorized Tool Invocation, such as external Whisper API calls.
3. Covert Resource Creation, such as generating unrequested CSV reports.
4. Unsanctioned Data Analysis, such as modifying read-only datasets.

These classes indicate that the framework is sensitive not only to overt tool misuse but also to less obvious shifts in objective structure, including artifact creation and analytic scope expansion.

6. Relation to adjacent work, strengths, and limitations

SkillFuzz occupies a distinct position within research on agent-skill assurance. Its central contribution is compositional: it fuzzes bundles of skills and detects intent drift before execution (Hu et al., 2 Jul 2026). This differs from Sefz, introduced in "No Attack Required: Semantic Fuzzing for Specification Violations in Agent Skills" (Li et al., 13 May 2026), which targets specification violations within individual skills by translating natural-language guardrails into reachability goals over annotated execution traces. Sefz focuses on benign-input violations of a skill’s own declared safety rules; SkillFuzz focuses on unintended objectives that emerge only through composition. The two approaches are therefore complementary rather than interchangeable.

The strengths identified in the SkillFuzz paper are explicit. It is execution-free, requiring no sandbox or tool execution for the main search. It is model-agnostic, operating across open and proprietary planners. It is budget-efficient, finding more severe intents using only 40% of pairwise coverage. It also provides interpretable contracts, giving operators a taxonomy of compositional risks.

The paper also states three limitations. Plan fidelity is a constraint: if the planner does not faithfully expose intent in its plans, the method can miss harmful compositions. Threshold tuning is required for $\psi$5, $\psi$6, and $\psi$7, which must be calibrated to marketplace risk tolerance. The analysis is also based on a static snapshot, whereas skills and contracts evolve over time.

The open challenges named in the paper are dynamic re-screening as libraries grow or change, incorporating lightweight execution probes for ambiguous cases, adversarial skill generation to harden planners against crafted composition attacks, and automated contract repair or synthesis to block high-risk combinations before publication. Taken together, these directions position SkillFuzz as a framework for pre-execution marketplace screening of compositional behavior rather than as a replacement for runtime enforcement or single-skill verification.