SkillDetonate: Runtime Skill Auditor

• SkillDetonate is a behavior-centric runtime auditor that detects malicious agent skills by monitoring OS-boundary events and dynamic data flows.
• It employs on-demand closure lift and marker-based taint propagation to counter multi-stage evasions and mitigate supply-chain risks.
• Evaluations show a 97% detection rate with low false positives, outperforming static scanners against adaptive malware tactics.

SkillDetonate is a behavior-centric, runtime auditor for third-party agent skills proposed in "Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malware" (Ji et al., 2 Jul 2026). It is designed for the software supply-chain risk created by public skill marketplaces, where a malicious skill executes with the agent’s privileges and can steal credentials, exfiltrate source code, or install backdoors. In contrast to install-time inspection, SkillDetonate executes a skill in a sandboxed LLM-agent session and detects malicious effects through OS-boundary information-flow evidence. The system combines on-demand closure lift with marker-based taint analysis, and the reported results show 97% attack detection at a 2% false-positive rate on a controlled benchmark, while sustaining 87% detection on real-world malicious skills (Ji et al., 2 Jul 2026).

1. Threat model and motivation

The motivating threat model is the third-party agent skill: a reusable capability obtained from a public marketplace and executed with the host agent’s privileges. Because such skills can access the agent’s execution context and surrounding system interfaces, they create a software supply-chain attack surface in which malicious code can target credentials, source repositories, or host integrity (Ji et al., 2 Jul 2026).

The paper situates SkillDetonate against existing skill defenses that rely on static scanning, including pattern matching and LLM-as-judge analysis. The central empirical claim is that it is unclear whether appearance-based defenses remain effective when an attacker preserves the payload’s behavior while changing its visible representation. To study this, the same work introduces SkillCloak, a payload-preserving evasion framework with two strategies: Structural Obfuscation, which rewrites visible payload indicators into semantically equivalent forms, and Self-Extracting Skill Packing, or SFS Packing, which hides malicious components from install-time inspection and restores them during agent execution (Ji et al., 2 Jul 2026).

Across eight scanners and 1,613 in-the-wild malicious skills, SFS Packing bypasses every scanner at over 90%, while Structural Obfuscation bypasses over 80% on most static scanners and reaches 96% on a hybrid scanner. The paper therefore argues that appearance-based auditing is insufficient under adaptive evasion. This motivates a runtime design that measures what a skill does at the OS boundary rather than how its files appear during installation (Ji et al., 2 Jul 2026).

2. System architecture

SkillDetonate operates as a sandboxed execution environment with three main components: a container-based sandbox, on-demand closure lift, and marker-based taint analysis (Ji et al., 2 Jul 2026). The sandbox hosts an agent adapter, exemplified by OpenCode + DeepSeek, that runs the skill exactly as the real agent would. It also hosts eBPF syscall tracing pinned to the container’s PID namespace and a FUSE-based virtual filesystem that interposes on every file read and write.

The FUSE layer serves two functions. First, it enforces the skill’s “closure” (Def. 4.1), which constrains the file view that the skill can act upon. Second, it injects synthetic data markers into sensitive reads. This design deliberately limits the observation surface to OS-level side-effects rather than source appearance. The pipeline described in Figure 1 is: Sandbox + Closure Lift + Taint Engine $\rightarrow$ Policy Violations $\rightarrow$ Malicious Verdict (Ji et al., 2 Jul 2026).

A concise architectural decomposition is as follows.

Component	Function
Container-based sandbox	Runs the skill in an LLM-agent session
eBPF syscall tracing	Observes process, file, exec, and network activity
FUSE virtual filesystem	Interposes on file I/O, enforces closure, injects markers
On-demand closure lift	Re-injects runtime-materialized instructions into the agent
Marker-based taint engine	Tracks sensitive-data flow across files, processes, context, and network

This architecture makes the detector agnostic to many syntactic payload transformations. A plausible implication is that the relevant unit of analysis is not the serialized skill artifact but the realized execution trace.

3. On-demand closure lift

On-demand closure lift addresses a specific failure mode of install-time and naive runtime inspection: multi-stage skills may unpack or generate their operative instructions only during execution. The paper highlights SFS-packed skills as the canonical case, where new SKILL.md files may be materialized after the agent session has already begun (Ji et al., 2 Jul 2026).

Mechanistically, the FUSE layer watches for newly materialized instruction files, such as *.md under the skill root. Whenever the skill writes a new SKILL.md or WORKFLOW.md into its closure, SkillDetonate immediately feeds the contents back into the same agent session so that the generated instructions are executed and traced. The pseudocode summary in the paper is a loop over FUSE events in which CREATE_FILE on a markdown file within skill_root triggers agent_session.inject_instructions(instr_text) before forwarding the event to the guest (Ji et al., 2 Jul 2026).

This component is presented as essential rather than auxiliary. Without closure lift, any skill that generates its real SKILL.md at runtime never executes that payload in the sandbox, and detection on packed variants collapses to approximately 30%. The stated significance is coverage of multi-stage workflows. In other words, closure lift converts runtime-discovered instruction material into first-class analysis input, preventing staged payloads from remaining semantically dormant during auditing (Ji et al., 2 Jul 2026).

4. Marker-based taint propagation and policy semantics

SkillDetonate’s second core mechanism is marker-based taint analysis. Sensitive-file reads, including examples such as ~/.ssh/* and ~/.aws/credentials, are intercepted by FUSE and do not return the real secret. Instead, they return a unique opaque data marker labeled #data_n. This preserves execution continuity while allowing the system to determine whether sensitive information would have propagated to an untrusted sink (Ji et al., 2 Jul 2026).

The taint model is defined over processes and inodes. Let $P$ be the set of processes and $I$ the set of inodes, with tainted subsets $T_p \subseteq P$ and $T_i \subseteq I$. Initialization is $T_i = \varnothing$ and $T_p = \varnothing$. For a syscall event read(p,i), if $i$ is a sensitive file, the system returns a marker and marks the marker inode as tainted; if $i \in T_i$, then $\rightarrow$0 becomes tainted. For write(p,j), if $\rightarrow$1, then inode $\rightarrow$2 becomes tainted. For exec(p,j), if $\rightarrow$3, then process $\rightarrow$4 becomes tainted because a tainted binary has been executed (Ji et al., 2 Jul 2026).

Propagation occurs on three planes. The first is an inode-level cross-process graph derived from eBPF: if a process reads a tainted inode, the process becomes tainted, and any inode subsequently written by that process becomes tainted. The second is in-process value taint via TaintStr and TaintBytes, which handles data encoded and sent within the same process. The third is LLM context taint: because the agent sees only #data_n, any later mention of that marker in generated text is also tainted (Ji et al., 2 Jul 2026).

Detection is expressed through two policy checks. The confidentiality policy forbids any data marker from reaching a network endpoint outside the trusted egress allowlist, $\rightarrow$5. The integrity policy restricts the skill, treated as a low-integrity principal, to writing or executing binaries only within allowlisted paths, $\rightarrow$6, such as its own root or approved install directories. On network_send(p, endpoint, data), a tainted process sending to an endpoint not in $\rightarrow$7 triggers a confidentiality violation. On fs_write(p, fpath), a tainted process writing outside $\rightarrow$8 triggers an integrity violation. The verdict rule is minimal: malicious if any violation is reported; otherwise safe (Ji et al., 2 Jul 2026).

The paper also reports an ablation-style rationale for the taint design. Without taint, SkillDetonate would flag every read and send as suspicious, increasing false positives to approximately 16%. Marker taint narrows reporting to flows in which sensitive data actually reaches an untrusted destination (Ji et al., 2 Jul 2026).

5. Evaluation and empirical performance

The reported evaluation uses two benchmarks. SkillJect contains 150 malicious and 50 benign skills and focuses on prompt-injection attacks in a controlled benchmark. MalSkillBench contains 622 in-the-wild malicious skills, described as malware verified to fire an observable effect in a Linux sandbox. The baselines are a Cisco static scanner, characterized as a regex+LLM hybrid, and a “Naïve Agent” detector that uses the same adapter stack and is asked, “Is this malicious?” (Ji et al., 2 Jul 2026).

On SkillJect, the headline numbers are as follows.

Detector	Malicious detected	False-positive rate
SkillDetonate	97%	2%
Cisco (static)	45%	11%
Naïve Agent	56%	14%

The paper states that SkillDetonate outperforms Cisco by +52 percentage points on detection and reduces false positives by half. It also emphasizes robustness under evasion: detection remains at 95–97% across raw, Structural-Obfuscation, and SFS-Packed variants, whereas Cisco falls to single digits, specifically below 25%, under packing (Ji et al., 2 Jul 2026).

On MalSkillBench, the results are more differentiated. For original skills, Cisco static reaches 98.6%, Naïve Agent 84.1%, and SkillDetonate 87.6%. For Structural-Obfused variants, the corresponding numbers are 10.1%, 79.4%, and 83.8%. For SFS-Packed variants, they are 9.6%, 56.6%, and 90.5%. Averaged across variants, Cisco static reports 39.4%, Naïve Agent 73.4%, and SkillDetonate 87.3% (Ji et al., 2 Jul 2026).

The overhead figures are also reported. SkillDetonate uses approximately 14 K tokens of LLM context per skill and approximately 153 seconds of wall time per skill, including multi-stage drive. Static scanners take approximately 20 seconds per skill and do not require a sandbox (Ji et al., 2 Jul 2026). This suggests a deliberate trade-off: higher runtime cost in exchange for resistance to payload-preserving evasions.

6. Interpretation, limitations, and relation to static scanning

The paper’s discussion frames SkillDetonate as a response to two specific weaknesses of static analysis. First, static, appearance-based scanners inspect only a subset of file types and artifacts. Second, Structural Obfuscation can rewrite or disperse malicious tokens and bypass regex or AST checks at least 80% of the time, while SFS Packing hides payloads in blind-spot directories and unpacks at runtime, evading at least 99.8% of static scans (Ji et al., 2 Jul 2026). SkillDetonate sidesteps these limitations by observing file writes, process execution, and network sends after the payload is operational, regardless of how the bytes were staged or transformed.

The discussion also clarifies two frequent misunderstandings. One is that runtime execution alone is sufficient; the reported collapse to approximately 30% detection on packed variants without closure lift shows that runtime auditing still needs explicit support for dynamically materialized instructions. The other is that broad behavioral monitoring alone guarantees precision; the reported rise to approximately 16% false positives without marker taint shows that behavior must be linked specifically to sensitive information flow (Ji et al., 2 Jul 2026).

Several limitations are explicitly acknowledged. Natural-language coverage gaps remain: if the agent refuses or fails to follow a malicious instruction, the payload never fires. The paper names a “force-execution” driver as a promising mitigation. Anti-sandbox logic embedded in skills can redirect execution away from malicious branches, in analogy to anti-VM malware, suggesting a role for hardened sandboxes and diverse execution environments. Finally, engineering constraints such as dead network sinks, missing API keys, and timeouts account for the approximately 13% of wild misses, though the paper states that these constraints do not invalidate the design (Ji et al., 2 Jul 2026).

Within this framing, SkillDetonate is best understood as a behavior-centric complement to install-time scanning rather than merely a stronger pattern matcher. The central claim is not that source appearance has no value, but that appearance alone is unstable under adaptive, payload-preserving evasions. The system’s contribution is to make the security decision depend on observed confidentiality and integrity violations at execution time.