SkillHarm: Risks in LLM-Agent Skill Ecosystems

• SkillHarm is a concept describing harmful outcomes induced by misused LLM-agent skills, where operational artifacts can trigger unsafe behaviors.
• It emphasizes context-dependent triggers, unsafe skill compositions, and lifecycle-aware attacks that emerge under specific runtime conditions.
• The research outlines detection, auditing, and mitigation measures including permission frameworks and runtime monitoring to secure agent behavior.

SkillHarm denotes harmful outcomes induced by agent skills as privileged, reusable artifacts in LLM-agent ecosystems. In recent work, the term spans several related phenomena: malicious behaviors embedded in a skill package; harms that emerge only when a skill is invoked under specific runtime conditions; retrieval failures that expose a query-specific risky sibling from the correct capability family; unsafe compositions of individually safe skills; and lifecycle-aware attacks in which a skill mutates persistent state and harms a later session rather than the current one (Ning et al., 1 Jun 2026, Lan et al., 10 Jun 2026, Ding, 9 Jun 2026, Wang et al., 30 May 2026). Across these formulations, the common premise is that a skill is not merely descriptive text: it is an operational bundle of instructions, scripts, resources, bindings, and execution assumptions that can steer agent reasoning and side effects with the agent’s privileges.

1. Skill artifacts, privileged trust, and the unit of analysis

A skill is treated in the literature as a loadable operational artifact. In OpenClaw- and ClawHub-style ecosystems, a skill typically consists of a directory whose primary SKILL.md provides metadata and procedural instructions, optionally accompanied by scripts, configuration, and auxiliary resources; agents discover installed skills through metadata and then read the full specification or execute referenced components at invocation time (Jiang et al., 16 Apr 2026, Ning et al., 1 Jun 2026, Chen et al., 15 Jun 2026). This packaging model makes skills closer to lightweight software extensions than to ordinary retrieved passages.

The security consequence is that the trust boundary is unusually porous. SkillGuard formalizes this by separating a context plane, which governs what a skill can load into the agent’s reasoning context, from an action plane, which governs side effects over files, processes, network, secrets, tools, and delegation (Pan et al., 2 Jun 2026). RouteGuard sharpens the distinction from traditional indirect prompt injection: skill poisoning is not anomaly insertion into neutral evidence, but malicious-instruction competition inside an instruction-bearing carrier that is already semantically licensed to guide agent behavior (Xiao et al., 24 Apr 2026).

The unit of risk is therefore not fixed. Some work treats the individual skill as the relevant security object, especially when the skill itself contains malicious or harmful functionality. Other work relocates the risk to the retrieved representative within a capability family, or to the installed set of co-resident skills. SkillResolve-Bench studies same-capability execution-risk retrieval, where a retriever selects the wrong sibling from the right family; SkillReact studies compositional risk, where each skill is individually safe but their capability union satisfies a forbidden pattern (Ding, 9 Jun 2026, Wang et al., 30 May 2026). SkillHarm is thus best understood as a family of failure modes tied to how skills are stored, retrieved, combined, invoked, and reused.

2. Mechanisms by which skills become harmful

A central finding is that harmful behavior is frequently context-dependent rather than statically obvious. Runtime Skill Audit describes several trigger surfaces: documentation that looks benign while helper scripts or resource rules reveal malicious branches only at execution; specific user requests that unlock hidden logic; local assets such as auth.json; persistent knowledge or memory surfaces; and multi-step tool interactions across filesystem, shell/runtime, web/network, sessions, or memory (Lan et al., 10 Jun 2026). In the paper’s case studies, a file-organizer becomes security-relevant only when local sensitive files exist, and a report-normalization workflow becomes harmful only when trigger words such as “continuity” or “archive” activate a branch that reconstructs a sensitive path and prepares an HTTP POST.

Lifecycle-aware attacks extend the threat beyond a single task execution. The SkillHarm benchmark defines Fixed-Payload Poisoning (FPP), in which a malicious payload is present at installation and can compromise the same session that invokes it, and Self-Mutating Poisoning (SMP), in which the first session appears benign but silently rewrites persistent skill content so that a later reuse is compromised (Ning et al., 1 Jun 2026). The benchmark organizes 12 risk types around three workflow components—data pipelines, system environments, and agent autonomy—including data exfiltration, output manipulation, poisoning, privilege escalation, unauthorized file modification, backdoor injection, denial of service, malware deployment, system corruption, goal hijacking, anti-forensics, and proxy attack.

A further escalation is dynamic malicious skills. DyMalSkill shows that a benign codebase can be weaponized when malicious natural-language content in SKILL.md induces the agent to edit the skill’s own code during execution (Chen et al., 15 Jun 2026). The attack relies on prerequisite framing, entry-script grounding, and wrapper-pierce formatting, with prepended payloads most effective because the skill is processed top-down. This shifts maliciousness from the distributed code artifact to documentation-induced runtime self-modification.

Other mechanisms do not require overtly malicious code at all. SkillMutator studies language-and-code cross-modal attacks, where risk emerges only from the joint interpretation of SKILL.md, executable files, and auxiliary resources; safe-looking helper code becomes harmful because the documentation frames it as optimization, reproducibility, warm-start logic, or maintenance (Kim et al., 12 Jun 2026). SkillResolve-Bench shows that harm can arise even when both candidates are ostensibly useful, if the retriever exposes a query-specific risky sibling with a stale resource binding, missing precondition, wrong procedure, stale API scope, or misleading example (Ding, 9 Jun 2026). SkillReact adds that two individually safe skills can compose into unsafe installed sets by satisfying forbidden conjunctions such as file-read plus network-out or downloader plus writer plus process-spawn (Wang et al., 30 May 2026). At the supply-chain level, repository-context analysis uncovered abandoned GitHub repository hijacking affecting 121 skills across two marketplaces, showing that SkillHarm also includes compromise of the hosting substrate rather than only the skill body (Holzbauer et al., 17 Mar 2026).

3. Taxonomies, benchmarks, and formal measurement

The literature operationalizes SkillHarm through several partially overlapping taxonomies. HarmfulSkillBench defines a Harmful Skill Taxonomy of 21 categories arranged into Tier 1 Prohibited Use (P1–P14) and Tier 2 High-Risk Use (H1–H7), then measures harmfulness over 98,440 skills from ClawHub and Skills.Rest (Jiang et al., 16 Apr 2026). Tier 1 includes categories such as cyber attacks, privacy violation, fraud and scams, platform abuse, and sexual content; Tier 2 covers areas such as legal advice, medical advice, and financial advice when safeguards are absent.

Lifecycle-aware measurement is organized differently. SkillHarm’s benchmark contains 879 attack samples across 71 skills, with 687 FPP samples and 192 SMP samples, each paired with a deterministic evaluator so that Attack Success Rate is measured as

$$\mathrm{ASR} = \frac{\#\text{ successful attacks}}{\#\text{ attempts}}.$$

It also defines conditional ASR (cASR), which conditions on the agent’s actual engagement with the poisoned file, and Attack Refusal Rate (ARR), which captures explicit refusal behavior (Ning et al., 1 Jun 2026). This separation matters because many nominal failures arise from non-engagement rather than genuine resistance.

Retrieval-centric work introduces different metrics. SkillResolve-Bench contains 661 helpful/risky pairs evaluated over a 7,982-skill fixed candidate pool, and reports helpful retrieval quality together with harmful sibling rate:

$$\mathrm{HSR@K} = \frac{1}{|Q|} \sum_{q \in Q} \mathbf{1}[r \in L_q^{(K)}].$$

Here the risky sibling is query-specific rather than intrinsically malicious, so the benchmark explicitly treats risk as an execution relation, not a permanent label on the skill (Ding, 9 Jun 2026).

SkillVetBench adds a two-stage vetting formulation. Stage 1 assigns a Skill Agentic Risk Score (SARS) using five integer dimensions—Instruction Fidelity Risk, Data Gravity, Action Irreversibility, Blast Radius, and Chain Amplification—aggregated as

$$\mathrm{SARS} = \frac{2\cdot d_{\mathrm{IFR}} + 1.5\cdot d_{\mathrm{DG}} + 1.5\cdot d_{\mathrm{AI}} + 2\cdot d_{\mathrm{BR}} + 2\cdot d_{\mathrm{CA}}}{2.7}.$$

The verdict mapping is Benign for SARS in $[0, 3.9]$, Suspicious for SARS in $[4.0, 6.9]$, and Malicious for SARS $\ge 7.0$, with runtime verification required before a final malicious verdict is issued (Hossain et al., 30 May 2026). HarmfulSkillBench, by contrast, evaluates model behavior using tier-specific composite scores:

$$\mathrm{Score}_{P} = (1-\mathrm{Refusal}) \times \frac{\mathrm{Harmfulness}-1}{4},$$

$$\mathrm{Score}_{H} = (1-\mathrm{Refusal}) \times \frac{\mathrm{Harmfulness}-1}{4} \times \frac{2-\mathrm{HiTL}-\mathrm{AID}}{2},$$

thereby separating prohibited-use refusal from safeguard compliance in high-risk domains (Jiang et al., 16 Apr 2026).

4. Detection and auditing paradigms

A recurring empirical conclusion is that artifact-level inspection alone is brittle. SkillVetBench reports that semantic-only and signature-based baselines are insufficient, missing up to 89% of malicious skills whose threats arise from natural-language instructions, multicomponent logic, or cross-component interactions (Hossain et al., 30 May 2026). Cloak-and-Detonate shows why: across eight scanners and 1,613 in-the-wild malicious skills, Self-Extracting Skill (SFS) Packing bypasses every scanner at over 90%, while Structural Obfuscation bypasses over 80% on most static scanners and reaches 96% on a hybrid scanner (Ji et al., 2 Jul 2026).

Runtime Skill Audit responds by shifting the decision boundary from artifacts to traces. Instantiated on OpenClaw and evaluated on 100 skills, RSA achieves 90.0% accuracy, 88.0% TPR, and 8.0% FPR, improving accuracy by 13.0 percentage points over the best static baseline; under self-evolving attacks, it continues to detect 19–20 out of 20 malicious skills across rounds (Lan et al., 10 Jun 2026). Its output labels are behavior-grounded—benign executed, harmful blocked, harmful executed, and uncertain—and are assigned from structured traces containing tool calls, file reads and writes, network requests, memory access, and refusal evidence.

SkillDetonate pushes runtime analysis deeper into OS-boundary information flow. By combining on-demand closure lift with marker-based taint analysis, it detects about 95%–97% of attacks at about 2% FPR on SkillJect and sustains about 87.3% detection on runnable wild malicious skills, while remaining robust under original, obfuscated, and packed variants (Ji et al., 2 Jul 2026). The key insight is that appearance-preserving evasions still have to realize sensitive reads, file writes, process creation, or network egress somewhere in the sandbox.

Install-time detection has also moved beyond lexical screening. RouteGuard measures attention hijacking and hidden-state alignment with a frozen backbone, reaching 0.8834 F1 on the critical Skill-Inject channel slice and recovering 90.51% of description-channel attacks missed by lexical screening (Xiao et al., 24 Apr 2026). SkillMutator, in turn, frames the problem as cross-modal reasoning and shows that a distilled Qwen2.5-Coder-7B-Instruct scanner improves detection on the strongest subset from 17.1% to 88.2%, surpassing GPT-4o-mini (23.7%) and GPT-5.4-mini (79.0%) and reaching frontier-level GPT-5.4 (86.8%) (Kim et al., 12 Jun 2026). Together these results partition the defense space into internal-signal pre-execution detection, cross-modal semantic scanning, trace-grounded runtime auditing, and information-flow verification.

5. Permission systems, isolation, retrieval controls, and selective invocation

Mitigation strategies increasingly treat skills as first-class principals rather than passive extensions. SkillGuard introduces a dual-plane permission framework with manifests, runtime access control, user-mediated authorization, deny-by-default enforcement, shell capability inference, and audit logging (Pan et al., 2 Jun 2026). On 315 real-world skills, its permission taxonomy covers 99.76% of observed protected objects, automated manifest generation reaches 91.0% F1, and attack success falls from 32.37% to 23.02% for contextual injections and from 25.56% to 16.67% for obvious injections while preserving benign task utility.

At the system boundary, DyMalSkill argues that skill source code should be immutable at runtime. Kernel-enforced read-only mounts implemented with Bubblewrap, plus a copy monitor that blocks copy-modify-run evasions, reduce ASR to 0.0% across all 12 behaviors on OpenHands/Qwen3.6-35B-A3B; all 300 benign skills still complete their tasks, and SHA-256 hashes remain unchanged pre- and post-execution (Chen et al., 15 Jun 2026). This defense is notable because benign skills rarely require source edits at runtime: in the paper’s measurement, only 1 of 300 benign skills modified code during execution.

Some mitigations operate before execution by constraining exposure rather than behavior. SkillResolve resolves candidate families, scores query-conditioned utility, and then selects exactly one representative per family before the final top-$K$ list (Ding, 9 Jun 2026). Under the released family relation, it reaches Recall@3 = 0.766, NDCG@3 = 0.699, and HSR@3 = 0, whereas removing representative selection leaves helpful ranking nearly unchanged but raises HSR@3 to 0.236. The safety mechanism is therefore not generic relevance improvement but within-family representative choice.

Operational guidance increasingly places runtime auditing inside deployment workflows. RSA recommends profile-guided runtime audits at skill submission, evidence-backed labels, re-audits after updates, CI/CD jobs that materialize sensitive contexts, and least-privilege runtime policies informed by behavior traces (Lan et al., 10 Jun 2026). SkillVetBench similarly emphasizes gating of high-risk primitives such as exec, write_file, install_skill, spawn, and subagent, because confirmed attacks are concentrated in these capabilities rather than in read-only tools (Hossain et al., 30 May 2026).

A broader, non-malicious reading of SkillHarm concerns mistimed or unnecessary invocation. SelSkill treats skill use as a learned skill-or-skip decision and reports that, on ALFWorld with Qwen3-8B, selective invocation improves task success by 10.9 percentage points and execution precision by 29.1 percentage points, while reducing average skill calls per episode from 2.55 to 0.44 by Round 3 (Chen et al., 30 May 2026). In this formulation, harmfulness is not adversarial content but context pollution, unmet preconditions, and procedural derailment caused by invoking a relevant skill at the wrong state.

6. Prevalence, ambiguity, and unresolved questions

Prevalence estimates vary sharply because the literature measures different objects. HarmfulSkillBench finds 4.93% harmful skills among 98,440 registry entries, with 8.84% on ClawHub and 3.49% on Skills.Rest (Jiang et al., 16 Apr 2026). SkillHarm cites recent evidence that 26.1% of public skills have at least one vulnerability (Ning et al., 1 Jun 2026). Repository-context analysis, however, argues that isolated skill scanners drastically overflag: in the context-scored subset, only 0.52% remained suspicious at the repository level, and the apparent malicious rate drops from 46.8% to 0.52% after incorporating repository congruence (Holzbauer et al., 17 Mar 2026). This suggests that prevalence is highly sensitive to whether one is measuring policy-violating functionality, latent vulnerability, scanner flags, or repository-level suspicion.

Risk is also conditional on retrieval, composition, and host-model behavior. SkillResolve-Bench shows that the same skill can be helpful for one query and risky for another, so labels may be query-specific execution relations rather than intrinsic properties (Ding, 9 Jun 2026). SkillReact estimates roughly 14,356 valid pair-pattern memberships in one registry after calibration and demonstrates that realization is model-gated: on an anchor-conditioned dropper subset, Haiku-4-5 issued the full download-then-execute chain on 36 of 39 direct-prompt trials, Opus-4-7 stopped at download-only, and Sonnet-4-6 refused outright (Wang et al., 30 May 2026). Installed skills therefore fix reachability, but the host model still decides whether the reachable chain is used.

Not all harmful outcomes are malicious in the supply-chain sense. In MCP-grounded offensive cybersecurity, “When Skills Don’t Help” reports only an 8.9 percentage-point spread from no-skills to full-skills, with $p = 0.71$ for the chi-squared test and a timing-side-channel setting in which richer skills degraded performance because false lesson propagation crowded out a better tactic (Chacko et al., 19 May 2026). This result complements SelSkill’s invocation-time findings: some forms of SkillHarm arise from procedural redundancy, overgeneralized lessons, or poor timing rather than from attacker intent.

Open problems remain consistent across the literature: rare triggers and long-horizon interactions are hard to cover; real external services are often replaced by shims; family relations and capability ontologies are incomplete; open-world retrieval pools differ from fixed-benchmark pools; and portability across agent ecosystems requires interface mapping, standardized taxonomies, and platform-specific instrumentation (Lan et al., 10 Jun 2026, Pan et al., 2 Jun 2026, Kim et al., 12 Jun 2026, Ji et al., 2 Jul 2026). The field’s central transition is from treating skills as static text or code artifacts to treating them as persistent, permission-bearing, composable, and behaviorally testable units whose risk depends on lifecycle state, retrieval context, installed neighbors, and host-model disposition.