Black-Box Skill Stealing Attack from Proprietary LLM Agents: An Empirical Study

Abstract: LLM agents increasingly rely on skills to encapsulate reusable capabilities via progressively disclosed instructions. High-quality skills inject expert knowledge into general-purpose models, improving performance on specialized tasks. This quality and ease of dissemination drive the emergence of a skill economy: free skill marketplaces already report 90368 published skills, while paid marketplaces report more than 2000 listings and over $100,000 in creator earnings. Yet this growing marketplace also creates a new attack surface, as adversaries can interact with public agent to extract hidden proprietary skill content. We present the first empirical study of black-box skill stealing against LLM agent systems. To study this threat, we first derive an attack taxonomy from prior prompt-stealing methods and build an automated stealing prompt generation agent. This agent starts from model-generated seed prompts, expands them through scenario rationalization and structure injection, and enforces diversity via embedding filtering. This process yields a reproducible pipeline for evaluating agent systems. We evaluate such attacks across 3 commercial agent architectures and 5 LLMs. Our results show that agent skills can be extracted with only 3 interactions, posing a serious copyright risk. To mitigate this threat, we design defenses across three stages of the agent pipeline: input, inference, and output. Although these defenses achieve strong results, the attack remains inexpensive and readily automatable, allowing an adversary to launch repeated attempts with different variants; only one successful attempt is sufficient to compromise the protected skill. Overall, our findings suggest that these copyright risks are largely overlooked across proprietary agent ecosystems. We therefore advocate for more robust defense strategies that provide stronger protection guarantees.

• The paper demonstrates that black-box attackers can recover proprietary agent skills in as few as three interactions, with baseline attacks averaging 48% exact-match and 95.11% embedding similarity across five commercial models.
• The authors generate 120 diverse attacks spanning roleplay, hypothetical, educational, chain-of-thought, and few-shot strategies, finding that few-shot prompts maximize semantic fidelity while chain-of-thought produces the highest exact-match recovery.
• SkillGuard-5 and LAN reduce exact-match extraction to 0%, but paraphrase, translation, and rewriting attacks retain substantial semantic leakage, showing that prompt and output filters do not fully protect proprietary capabilities.

Motivation and threat model

LLM agent systems increasingly package reusable capabilities as skills—structured artifacts containing metadata, workflow instructions, and supporting resources, typically stored as a SKILL.md file that is loaded at runtime. The commercial value of these artifacts is substantial: the paper cites a public dashboard reporting 90,368 published skills and 24.3M cumulative installs on skills.sh, alongside paid marketplaces such as Claw Mart with 2,000+ listings and over $100,000 in creator earnings. This monetization creates a direct incentive for unauthorized extraction. Unlike system prompts, skills are modular, structured, and directly tied to marketplace distribution, so leakage constitutes theft of a transferable capability rather than mere prompt disclosure.

The paper formulates skill stealing as a black-box attack in which an adversary interacts only through a public agent interface or API, issues arbitrary multi-turn natural-language queries, and attempts to recover the full SKILL.md content—verbatim or in a semantically equivalent transformed form. The adversary has no access to model weights, backend configuration, or local skill files; the defender controls system instructions, skill injection into context, and input/output filtering. The leakage path is straightforward: a crafted prompt causes the agent to load the hidden skill file during execution, after which the model reproduces its content in the response.

Automated attack generation framework

The authors derive an attack taxonomy from prior system-prompt-stealing work along two dimensions: scenario construction (Educational, Hypothetical, Roleplay, or None) and structure injection (Chain-of-Thought, Few-Shot Demonstration, or None), yielding $4 \times 3 = 12$$strategy combinations. Ten <a href="https://www.emergentmind.com/topics/gpt-5" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">GPT-5</a>.4-generated seed prompts are instantiated under each combination, producing a benchmark of 120 extraction prompts. Diversity is enforced via embedding-based filtering using text-embedding-3-small: candidates whose maximum cosine similarity to the existing pool exceeds$$\tau = 0.75$$are discarded and regenerated.</p> <p>Evaluation uses four complementary metrics: exact-match containment (EM), ROUGE-L, embedding cosine similarity, and an LLM-as-a-judge leakage ratio in$$[0,1]$$. Each prompt is run against the target agent (OpenCode) over up to three attempts, scoring the response that maximizes EM or otherwise ROUGE-L. Five commercial models are tested: GPT-5, MiniMax-M2.7, Kimi-k2.5, <a href="https://www.emergentmind.com/topics/deepseek-v3-2" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">DeepSeek-V3.2</a>, and Claude-haiku-4.5, targeting the high-usage <code>find-skills</code> skill among eleven installed skills.</p> <h2 class='paper-heading' id='attack-results'>Attack results</h2> <p>The central empirical finding is that extraction succeeds with minimal effort—the abstract reports full skill recovery in as few as <strong>3 interactions</strong>. Even the unmodified baseline$$(NO, NO)$ averages 48% EM, 79.95% ROUGE-L, 95.11% cosine similarity, and a 0.91 LLM-leakage ratio across models. Strategy effects are metric-dependent:

Strategy	EM	ROUGE-L	Cosine	LLM-leakage
(None, None)	48%	79.95%	95.11%	0.91
(None, Few-Shot)	52%	90.74%	97.34%	0.93
(None, CoT)	72%	75.50%	95.09%	0.95

Few-shot demonstrations most reliably preserve lexical and semantic fidelity of the stolen content, whereas Chain-of-Thought elicits more explicit disclosures but perturbs output format. Model heterogeneity is large: GPT-5 reaches 90% EM under the baseline alone, Kimi-k2.5 attains 100% EM under several CoT settings, while DeepSeek-V3.2 fails baseline EM yet leaks strongly under few-shot variants—indicating that single-metric evaluation understates exposure. Claude-haiku-4.5 is the most strategy-sensitive, dropping sharply under roleplay combined with CoT.

Cross-platform experiments confirm generality. On Claude Code (MiniMax-M2.7), EM ranges from 10–60% with very high ROUGE-L and cosine similarity; on Codex (GPT-5), EM is zero due to encoding errors in outputs, but removing anomalous characters yields content consistent with the original SKILL.md, and semantic metrics remain high (e.g., LLM-leakage ratio 0.99 in many cells). Case studies further show successful extraction from the ChatGPT web interface (GPT-5.4) and the Claude web platform (Sonnet-4.6), demonstrating that the threat transfers beyond local agent frameworks to frontier commercial stacks. Ablations across target skills show strategy-dependent vulnerability: azure-ai exhibits nearly saturated EM under most settings, while others favor few-shot or CoT depending on the metric.

Defenses

The paper evaluates defenses at three pipeline stages.

Input-phase detection. An LLM-based binary classifier screens queries before they reach the agent. On a balanced set of 120 stealing prompts and 120 benign find-skills-related prompts, it achieves TPR = 1.00, FPR = 0.00, and F1 = 1.00. This perfect result should be read cautiously: positives come from the same generator as the training intuition behind the detector's design, so generalization to out-of-distribution attack phrasings remains untested.

Inference-phase context hardening. Beyond instruction and sandwich baselines, the authors propose SkillGuard-5, a five-component defensive prompt covering boundary constraints, hierarchy rules, equivalence constraints (treating paraphrases, translations, and summaries as prohibited), refusal policy, and utility preservation. SkillGuard-5 reduces EM to 0% across all scenario–structure combinations—a strong result—but residual semantic leakage remains non-trivial (e.g., average ROUGE-L up to ~70% under few-shot attacks on some models). Instruction defense leaves 18% average EM under the baseline and sandwich defense 22%, confirming that simple refusal-style prompting offers limited protection.

Output-phase LAN defense. LAN combines an LLM-leakage-ratio threshold ($\tau_{\mathrm{LLM}} = 0.85$) with NVRecall, a paragraph-level approximate-matching recall measure robust to formatting changes, thresholded at $\tau_{\mathrm{nv}} = 0.30$. Filtering reduces EM to 0% in every setting with FPR = 0.00 throughout; under the baseline, ROUGE-L, cosine similarity, and LLM-leakage drop from 80%, 95%, and 0.91 to 2%, 14%, and 0.05. The hardest case, (Hypothetical, CoT), retains filtered scores of 17% ROUGE-L, 71% cosine similarity, and 0.39 leakage ratio.

A defense-comparison study exposes a critical weakness: translation- and rewrite-based reconstruction attacks preserve substantial semantic leakage (ROUGE-L/cosine/LLM ratios often 0.7–0.95) even when EM is near zero, and the LAN thresholds prove too permissive under these transformation-heavy conditions—filtered outputs remain nearly identical to originals. Exact-match filtering is the most reliable rule here, but it presupposes access to the reference text and strict string comparison.

Limitations and open questions

Several caveats bear directly on the results. First, the defense evaluations use small, generator-correlated datasets (240 prompts for input detection; benign sets derived from the same model), so reported perfect detection likely does not reflect adaptive adversaries who deliberately evade the detector's known semantics—an issue the authors themselves acknowledge by noting that repeated automated attempts make any single-point defense insufficient, since one success compromises the skill. Second, the LAN defense requires the defender to possess the protected skill text for comparison, which is feasible for proprietary skills but not for third-party marketplace content. Third, the finding that SkillGuard-5 eliminates EM but not semantic leakage leaves open whether any prompt-level defense can bound paraphrased disclosure at all. Fourth, evaluation covers one primary target skill plus ablations on ten others within one agent framework; how leakage rates generalize to skills with different structures, lengths, or injection mechanisms is unresolved. Finally, the Codex encoding anomaly shows that surface-form metrics can mischaracterize leakage, motivating more robust canonicalization in future measurement.

Conclusion

This study establishes skill stealing as a practical, inexpensive, and broadly effective black-box attack against proprietary LLM agents, demonstrated across five models, three commercial architectures, and two frontier web platforms. Direct prompts already achieve near-saturated semantic leakage, and lightweight defenses—while reducing exact-match recovery to zero—leave meaningful residual semantic exposure, particularly under transformation-based reconstruction. The results indicate that copyright risks in emerging skill marketplaces are largely unaddressed by current deployment practice, and that stronger protection guarantees beyond prompt-level filtering will be necessary as skills become a primary unit of monetizable agent capability.