- The paper evaluates DeepSeek Harness with 14,560 real-runtime executions across 16 content channels, two carrier modes, and 13 attack methods, using A.I.G., RuleJudge, and LLMJudge to trace attacker-controlled content from sources to sensitive sinks.
- The paper finds 5.6% full success and 7.6% broad influence under RuleJudge, while LLMJudge records 5.3% full success and 12.6% broad influence, with fake completion, hidden Unicode, and skills channels producing the highest-risk results.
- The paper recommends preserving source provenance, independently authorizing sensitive tool calls, managing skills and integrations as code-adjacent assets, and rerunning source-to-sink tests after changes to models, parsers, prompts, or policies.
Overview
This report from Tencent Zhuque Lab presents a controlled security assessment of indirect prompt injection in DeepSeek Harness (DSH), an open-source, plugin-based agent framework, using the AI-Infra-Guard (A.I.G) testing framework (2608.16393). The assessment executes DSH's real TypeScript agent loop rather than a synthetic substitute, delivering attacker-controlled payloads through content-reading source tools and observing whether the agent attempts sensitive sink actions. Across 14,560 runtime executions spanning 16 content channels, two carrier modes, and 13 attack methods, the study identifies high-risk slices—fake completion at 17.0%, hidden Unicode in file mode at 25.5%, and the skills channel in file mode at 16.0% full success under the LLM-based judge—that aggregate rates alone would obscure.
Threat model and evaluation target
The assessment adopts a source-to-sink model of indirect injection. A source is a content-reading tool whose result includes attacker-controlled material (a tainted artifact); a sink is an externally observable action the attacker wants the agent to perform, such as email submission, command execution, or fund transfer. The attacker controls the tainted artifact but not the benign user request, system prompt, tool registry, judge, or sink implementation. All sinks are local fixtures that log calls and arguments without real side effects, so a recorded sink call represents an attempted sensitive action rather than an actual one. The paper is explicit that the percentages are descriptive for this controlled configuration, not claims of universal vulnerability rates for DSH or its model provider.
DSH is chosen as a target because its plugin architecture—model adapter, tool registry, session log, and agent loop composed from configuration—makes the relevant trust boundaries explicit while creating many routes through which untrusted content becomes model-visible.
Assessment method and runtime integration
A.I.G constructs the indirect-injection test matrix, injects tainted artifacts via a taint router into source tools, drives the real DSH runtime through a Python-to-TypeScript adapter, and records normalized source-to-sink traces. Each trace contains the user request, model messages, tool calls, tool results, taint and sink annotations, and final response. The trace is the unit of analysis, allowing reviewers to distinguish quoting a malicious instruction from planning or attempting an action.
Two complementary judges assign outcomes. RuleJudge is a deterministic evaluator checking taint reach, sink invocation, critical argument match, canary dominance, and refusal detection. LLMJudge is a semantic evaluator that reads the complete trace against the task-specific criterion and returns Full, Partial, or Failure without overwriting the RuleJudge verdict. Full success requires a sink call with expected arguments (for sink-required tasks) or a canary-dominated short response (for output-only tasks); mismatched critical arguments or quoted canaries yield partial compliance.
The dataset (full_channel_mode) contains 1,120 balanced base cases over 16 channels and two carrier modes with 35 payload objectives (32 sink-required, 3 output-only). Applying a naive unmodified baseline plus 12 attack methods yields 14,560 executions. The backend is DeepSeek-V4-Flash through a local proxy, with no A.I.G prompt-hardening transformation enabled—a deliberate scoping choice the authors state plainly. Text mode uses textual carrier representations; file mode constructs real formats (HTML, email, calendar, PDF metadata, spreadsheets) and extracts model-visible material, thereby testing parsing, metadata, and encoding effects that text-only approximations omit.
Code-grounded analysis of the DSH injection path
Inspecting the DSH source snapshot (commit 47f943859bef), the authors identify two control-flow locations relevant to injection. First, the agent-loop tool-call module appends tool results to the session and accepts additionalContexts returned by those results, with deferContext exposed in the tools interface. These are normal composition mechanisms, but they mean any retrieval tool, MCP integration, skill, or plugin that controls result content participates in the model-visible input boundary; source provenance must remain available to later policy decisions. Second, DSH defines a monotonic ToolGuard executed after tools/pre-execute listeners and before the tool body, where returning a reason denies the call and later listeners cannot convert a denial into an allow. The authors are careful to state that the experiment does not show these interfaces are flawed; it shows why deployments must use them when an untrusted source precedes a sensitive sink.
Results
Overall, RuleJudge assigns 819 of 14,560 runs (5.6%) to full success and 298 (2.0%) to partial compliance, for a broad-influence rate of 7.6%. LLMJudge assigns 772 (5.3%) full and 1,060 (7.3%) partial, for 12.6% broad influence. The divergence is concentrated in partial compliance, which the semantic judge labels far more often—capturing quoted canaries, incomplete sink arguments, and traces where planning changed without meeting the full criterion. The authors argue the disagreement is informative rather than incidental: RuleJudge's exact criteria suit regression testing, while LLMJudge should prioritize traces for manual review rather than collapsing the two into a single score.
| Judge |
Full success |
Partial |
Broad influence |
| RuleJudge |
819 (5.6%) |
298 (2.0%) |
7.6% |
| LLMJudge |
772 (5.3%) |
1,060 (7.3%) |
12.6% |
Carrier representation is part of the attack surface. File mode produces a higher RuleJudge full-success rate than text mode (6.2% vs. 5.1%) and more observed sink calls (387 vs. 254), though the aggregate difference does not hold for every channel.
Attack methods. The naive baseline reaches 5.5% RuleJudge full success in text mode and 5.9% in file mode. Fake completion is the strongest method, reaching 16.6% (RuleJudge) and 17.0% (LLMJudge) in text mode. Obfuscation shows the sharpest judge disagreement: 13.6% under RuleJudge in both modes but only 9.1% and 8.8% under LLMJudge, consistent with traces where a mechanical signal occurred without complete semantic compliance. The authors caution that a rate above naive indicates increased exposure in this specific configuration, not a universal ordering of methods.
Channels. The skills channel is elevated in both modes (14.3% text, 16.0% file, LLMJudge). The largest individual cell is hidden Unicode in file mode at 25.5% (116 of 455 runs), against 0.0% in text mode. This contrast is the paper's strongest evidence that file-mode testing is necessary: replacing a real carrier with plain text removes the parsing and representation behavior that determines whether the payload reaches the model at all. The skills result also places reusable agent instructions and integrations—normally treated as trusted assets—inside the prompt-injection threat model, implying they require ownership, provenance, version review, and privilege restrictions.
Deployment implications
The authors argue the vulnerability path is systemic rather than a weakness in one prompt or tool, spanning ingestion, carrier parsing, tool-result serialization, session construction, planning, and tool-call authorization. They derive four controls: preserve provenance at the model boundary (source labels, trust tiers, normalization exposing hidden Unicode and metadata); authorize sensitive sinks independently of model interpretation (allowlists, argument-level checks, user approval); treat skills, MCP integrations, and tool descriptions as code-adjacent assets with lifecycle controls; and rerun the source-to-sink matrix after any change to prompts, tools, parsers, model providers, or authorization policy.
Limitations and open questions
The paper is explicit about its scope: one DSH revision, one model backend (deepseek-v4-flash), one agent persona, a baseline configuration with no hardening enabled, and simulated sinks with no real-world side effects. Attack-method rankings are configuration-specific, and the study does not compare boundary defenses such as spotlighting, BIPIA, or StruQ—it only identifies where such controls could be applied in DSH. Open questions include whether the observed channel and method rates transfer to other model backends and harness configurations, whether the RuleJudge/LLMJudge disagreement can be resolved into a principled combined verdict, and whether source-aware policy enforced through DSH's ToolGuard and pre-execute hooks measurably reduces the high-risk slices reported here.
Conclusion
This report demonstrates a repeatable methodology for assessing indirect prompt injection against an unmodified agent runtime, combining real-runtime execution, a broad carrier-and-wording matrix, trace-level evidence, and dual deterministic/semantic judging. Its principal findings are that carrier representation materially changes exposure (hidden Unicode: 25.5% in file mode, 0.0% in text), that agent-specific channels such as skills are high-risk injection vectors, and that deterministic and semantic judges measure different classes of failure that should be reported separately. The results are descriptive of one controlled configuration, but they concretely identify where DSH deployments can enforce source-aware policy before sensitive sinks execute.