Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malware

Abstract: LLM coding agents increasingly rely on third-party agent skills from public marketplaces, which execute with the agent's privileges and create a software supply-chain attack surface: a malicious skill can steal credentials, exfiltrate source code, or install backdoors. Existing defenses use static skill scanners based on pattern matching or LLM-as-judge analysis, but it remains unclear whether they withstand adaptive evasions that preserve malicious behavior while changing payload appearance. This paper first presents an adversarial study of existing skill scanners through SkillCloak, a payload-preserving evasion framework that keeps the attack semantics intact while transforming their visible form. SkillCloak uses two complementary strategies: Structural Obfuscation, which rewrites visible payload indicators into semantically equivalent forms, and Self-Extracting Skill (SFS) Packing, which hides malicious components from the install-time view and restores them during agent execution. Across eight scanners and 1,613 in-the-wild malicious skills, SFS Packing bypasses every scanner at over 90%, while Structural Obfuscation bypasses over 80% on most static scanners and reaches 96% on a hybrid scanner, showing that appearance-based auditing is insufficient. Motivated by this finding, we propose SkillDetonate, a behavior-centric runtime auditor that executes skills in a sandbox and detects malicious effects through OS-boundary information-flow evidence rather than install-time appearance. SkillDetonate combines on-demand closure lift, which observes instructions materialized during execution, with marker-based taint analysis, which tracks sensitive-data flows across the agent context, files, processes, and network operations. The results show that SkillDetonate detects 97% of attacks at a 2% false-positive rate and sustains 87% detection on real-world malicious skills.

• The paper proposes SkillCloak for evading static analysis and SkillDetonate for behavior-centric, dynamic detection of malicious agent skills.
• It demonstrates that SkillCloak achieves over 90% bypass rates—including up to 99.9% for static scanners—using structural obfuscation and SFS packing.
• SkillDetonate leverages OS-level taint analysis to dynamically detect malicious actions, achieving 97% detection on controlled benchmarks.

SkillCloak and SkillDetonate: Scanner Evasion and Dynamic Detection of Agent Skill Malware

Introduction and Problem Setting

The proliferation of LLM-based coding agents has resulted in the emergence of modular “agent skills,” composable packages distributed via public marketplaces that LLM agents can load dynamically to gain new functionalities. This extensibility, however, introduces a supply-chain threat vector: a skill, once installed, executes with the agent’s privileges, enabling credential theft, source exfiltration, and backdoor insertions by malicious actors. Existing defenses focus on install-time static analysis—either through pattern matching or LLM-based review—but their robustness against adversarial, semantics-preserving transformations remains underexplored.

Figure 1: Conceptual overview of SkillCloak (offensive evasion) and SkillDetonate (behavior-centric defense).

Design and Evaluation of SkillCloak

Evasion Methodology

SkillCloak operationalizes two complementary evasion strategies:

1. Structural Obfuscation: This method rewrites payload signatures (e.g., command tokens, credential paths, URLs) to evade pattern/AST/LLM-based detection without modifying the executable payload. Obfuscations include unicode confusables, casing, sharding of tokens across files/lines, extension manipulation, and leveraging shell/Python dynamic resolution. The goal is to minimally perturb the skill structure while maximizing scanner bypass.

   Figure 2: Workflow of Structural Obfuscation, transforming payloads while retaining their semantics.

2. Self-Extracting Skill Packing (SFS Packing): This approach removes the entire malicious payload from the scanner’s semantic scope by moving it into directories/files that scanners ignore and installing a benign-appearing cover plus a decoder stub. Upon execution, the agent decodes and installs the full malicious payload in-memory, defeating all install-time static analyses.

   Figure 3: SFS Packing encapsulates the original malicious workflow inside an install-time opaque blob, only extracting at runtime.

Adversarial Evaluation

Across a corpus of 1,613 in-the-wild, ground-truth-malicious skills curated from a public marketplace, SkillCloak is shown to break the defense-in-depth provided by eight real-world scanners (including hybrid and LLM-as-judge designs). SFS Packing achieves over 90% bypass against every scanner, with several static scanners surpassed at 99.8–99.9%. Structural Obfuscation yields 80–98% bypass rates, with hybrid models sometimes even more vulnerable.

Figure 4: Per-scanner bypass rate for SkillCloak across static, hybrid, and semantic/LLM-based auditors.

Notably, none of these evasive transformations impair the underlying utility or functionality of cloaked skills: systematic evaluation with production agents (OpenAI Codex and Claude Code) on a standardized multi-skill benchmark shows no statistically significant decrease in execution success rates or trajectory similarity.

Implications

These empirical results demonstrate a critical limitation of static skill auditing, especially when faced with dedicated adversaries. The contradiction is explicit: static and even LLM-based scanners are systematically circumvented by semantics-preserving disguise, yet the skills remain fully functional and thus operationally dangerous.

SkillDetonate: Dynamic, Behavior-Centric Detection

Motivating a Shift to Behavior

Given the demonstrated ineffectiveness of static and surface-level semantic checks, the detection paradigm must transition from appearance- to effect-based auditing. SkillDetonate is introduced as a dynamic auditor, focusing on OS-level information flow, specifically what a skill does during execution rather than its install-time appearance.

Figure 5: SkillDetonate's architecture: skill sandboxing, on-demand closure execution, and OS-level information-flow tracking via taint analysis.

Design Components

• On-Demand Closure Lift: Skills are not bound to their package at rest; they may dynamically generate new executable resources (e.g., SKILL.md) upon unpacking/decoding. SkillDetonate’s FUSE-backed mechanism recursively re-ingests and executes any such runtime materializations, ensuring multi-stage payloads are always detonated.
• Marker-Based Taint Analysis: Fine-grained, multi-plane tracking of sensitive data flows is performed. Marker tokens are injected during intercepted reads (e.g., credential pulls), and their propagation is observed across process boundaries, file/network exports, and agent context. Edge flows are reconstructed using eBPF syscall traces, and integrity/confidentiality violations are flagged if markers reach unauthorized sinks or destinations.

Detection Performance

On SkillJect (a prompt-injection-focused benchmark) and MalSkillBench (703 executable wild skills), SkillDetonate identifies 97% of attacks at a 2% false-positive rate in controlled settings and achieves 87.3% detection on wild samples, remaining robust to all SkillCloak evasions. Static and LLM-based baselines, in stark contrast, degrade by up to 90% under SFS Packing or structural obfuscation. False positives in benign skills are minimal due to precise information-flow tracking.

Failure analysis finds the primary root cause of missed malicious samples in the wild is NL agent coverage—that is, behaviors not triggered by the agent during sandboxed execution, not detector limitations.

Figure 6: Root cause analysis for missed detection in the wild, dominated by agent execution/instruction coverage gaps.

Ablation and Robustness

Ablation experiments confirm SkillDetonate’s two architectural novelties are individually load-bearing. Omitting on-demand closure lift eliminates detection of multi-stage packed skills; disabling taint analysis results in an eightfold surge in false positives. Notably, SkillDetonate also detects a broad class of independent evasions (VulMask), establishing generality.

Theoretical and Practical Implications

SkillCloak codifies the fundamental undecidability of perfect static detection in environments where the adversary can mutate the representational surface without altering underlying behavior, echoing classical results in malware research. SkillDetonate positions dynamic, behavior-centric analysis—rooted in OS-level information flow—as the practical trust gate for LLM agent ecosystems.

However, agent-driven execution leaves open the problem of natural-language coverage: malicious paths contingent on agent decisions or environmental state can evade detonator-driven traces. This challenge is analogous to forced execution in binary analysis and motivates the development of force-execution agents for NL-branch path coverage.

Static scanners, while insufficient alone, retain utility as hygiene prefilters. Structural hardenings such as post-install tree integrity attestation or blocking opaque/blob assets may marginally raise attacker cost but do not address coverage gaps inherent to the install-time paradigm.

Conclusion

This study demonstrates with strong empirical evidence that static and hybrid skill scanners are fundamentally inadequate against semantics-preserving evasions in the agent skill supply chain. SkillDetonate—by leveraging dynamic, behavior-centric information-flow auditing—achieves robust detection of both original and evasive skill malware. Further advances in agent instrumentation, force-execution for NL branches, and standardized sandboxes are warranted to fully close the execution coverage gap in next-generation LLM agent ecosystems.