- The paper introduces Cordon-MAS, a compartmentalized multi-agent architecture that isolates raw evidence, audits structured claims, and gates synthesis, reducing average attack success from 27.5% to 2.1%.
- The paper shows that prompt-based monitoring leaves a control gap—models can detect contradictions yet endorse poisoned claims—while Cordon-MAS achieves near-zero attack success across multiple backends and datasets.
- The paper identifies coordinated multi-source poisoning as the key remaining weakness, with adaptive attacks reaching roughly 26% attack success, while the defense also incurs 2.2× latency, 2.8× cost, and lower answerability.
Motivation and the monitoring-control gap
Retrieval-augmented generation (RAG) systems are vulnerable to knowledge poisoning, in which adversarially optimized documents injected into a corpus manipulate generated answers. The most capable attack considered here is Confundo-style poisoning, in which a poison generator is optimized end-to-end so that malicious documents survive preprocessing, reranking, and paraphrasing—the transformations that simpler defenses rely on. Existing defenses (filtering, activation-based detection, trust scoring, isolation) share an implicit assumption: if poisoned evidence can be identified, the system will avoid acting on it.
The paper's central empirical finding is that this assumption fails. The authors identify a monitoring-control gap: models can detect contradictions and suspicious evidence in their reasoning traces yet still endorse poisoned claims in the final output. Detection is monitored but not enforced at the point of action commitment. The implication is that poisoning is not fundamentally a detection problem but an information-flow control problem: as long as untrusted natural-language evidence can directly condition the final generator, an optimized poison can exploit instruction-following to control output.
The Cordon Principle and architecture
The paper proposes the Cordon Principle: no agent capable of final natural-language synthesis may access untrusted natural-language evidence. Cordon-MAS realizes this principle through a compartmentalized multi-agent pipeline with asymmetric memory privileges, enforced architecturally rather than via prompting. Three falsifiable invariants define the guarantee:
- I1 — Dirty-read isolation: the Synthesizer never sees raw document text; its only inputs are certified structured claim cards.
- I2 — Claim-only communication: inter-agent messages are structured triples (entity, relation, object, source, rank, confidence, risk score), confining raw evidence to the Extractor.
- I3 — Certified synthesis: a claim reaches the Synthesizer only after extraction, passing the Auditor's risk threshold, and a Gate answerability declaration.
The pipeline comprises an Extractor (the sole reader of raw documents), an Auditor that scores each claim via a risk function combining marginal influence I(ci) and cross-source support: R(ci)=I(ci)⋅(1−S(ci)), rejecting claims with R>0.65; a Gate providing an independent second blocking layer on answerability; and a Synthesizer restricted to Gate-approved claims. The paper frames the security guarantee as verifiable by inspecting state transitions, converting the design philosophy into checkable conditions.
The authors also offer an informal mechanistic explanation, the Attention Contamination Observation: in an autoregressive transformer, hidden states at all positions following a poison token are convex combinations of value vectors including poison contributions, and a model cannot inspect a token without attending to it. Prompt instructions can attenuate this channel but cannot guarantee zero attention allocation to poison positions; architectural isolation severs it entirely. The authors are explicit that this is an informal observation, not a formal theorem, since proving impossibility would require a complete characterization of instruction-following in transformers. The empirical gap—prompt-based defense at 24% ASR versus Cordon-MAS at 0.0%—is presented as the substantive evidence.
Experimental results
Evaluation uses five BEIR datasets (SciFact, FiQA, NQ, MS MARCO, HotpotQA; 50 queries each, with n=100 validation on two datasets), Contriever retrieval (K=10), DeepSeek-Chat as the primary backend, and the Confundo threat model with single- and multi-document injection.
| Method |
SciFact |
FiQA |
NQ |
MS MARCO |
HotpotQA |
Avg. |
| Vanilla RAG |
62.0% |
18.0% |
8.2% |
20.9% |
28.6% |
27.5% |
| Paraphrase |
58.0% |
26.0% |
10.2% |
16.3% |
30.6% |
28.2% |
| TrustRAG |
60.0% |
14.0% |
8.2% |
23.3% |
24.5% |
26.0% |
| RobustRAG |
44.0% |
10.0% |
2.0% |
9.3% |
6.1% |
14.3% |
| Debate |
38.0% |
12.0% |
6.1% |
11.6% |
16.3% |
16.8% |
| Cordon-MAS |
2.0% |
4.0% |
0.0% |
4.7% |
0.0% |
2.1% |
Cordon-MAS achieves a 92.4% relative ASR reduction over undefended RAG, with a pooled Wilson 95% CI of [0.8%, 4.7%] non-overlapping with any baseline. An independent seed replication (mean ASR 0.8%) confirms rank stability. Cross-backend validation on GPT-4o and Qwen2.5-32B yields near-identical ASR (0–6%), supporting the claim that the defense is architectural rather than model-specific. Notably, GPT-4o exhibits a double-edged sword effect: its stronger instruction-following raises unprotected vanilla ASR to 52% (SciFact) and 38% (NQ), yet it performs comparably under Cordon-MAS.
The monitoring-control gap is validated directly with prompt-based baselines on the same backend: CoT-Detect reduces ASR from 34.0% to 24.0% but the model still endorses poison in 24% of queries despite flagging contradictions in its reasoning traces, with an 82% "I don't know" rate on poison queries indicating excessive caution. The two-stage Danger Evaluator reaches 10.0% ASR at 2× API cost. Neither reaches zero, whereas Cordon-MAS achieves 0.0% on the same datasets. The paper frames this as channel removal versus behavioral attenuation: prompt defenses leave the generator in the same information-flow regime as vanilla RAG.
Ablation shows the Auditor is the most critical component (4–16× ASR increase when removed; mean 7.1×), with the Gate a secondary layer (mean 5.2×); the monotonic ordering full < no_gate < no_auditor holds across all datasets and both seeds. The layered cascade rejects 86% of poison claims at audit and blocks 67% of remaining poison queries at the Gate. A Pareto analysis across poison density K∈{1,…,5} shows Cordon-MAS tracing the left frontier (ASR ≤ 2% at all K) while vanilla RAG spans 39–98% ASR; the paper argues the 60% clean-utility cost is revealed by the defense rather than created by it.
Utility trade-off
Cordon-MAS answers 60% of clean queries on average (40–79% per dataset), with correctness on answered queries of 78–88% versus 46–67% for vanilla RAG. The net correct-answer rates are comparable (~49% vs. ~53%), but the failure modes differ: Cordon-MAS explicitly refuses 40% of queries—refused queries are never wrong—while vanilla RAG silently errs on 47% with no uncertainty signal. The paper also documents that TrustRAG's refusal behavior is poorly calibrated, blocking more clean queries (27% average) than poison queries (12%). A notable transparency point: the initial configuration answered only 4–14% of clean queries at 3.5% correctness; three prompt refinements (query-aware extraction, relaxed Gate threshold, Synthesizer trust calibration) raised utility to 60% without architectural change, which the authors cite as evidence that the security guarantee does not depend on prompt tuning.
Adaptive attacks and generalization
Three adaptive strategies probe the defense's audit mechanism. Consistency collusion—multiple poison documents fabricating mutually corroborating claims—achieves 70.3% audit bypass and 63.3% pipeline penetration, defining the primary security boundary: when the attacker controls enough sources, cross-source consistency becomes a liability rather than a safeguard. Judge confusion (57.3% bypass) and claim mimicry (31.5%) are weaker. Under adaptive attacks, Debate ASR rises 2–3× above its naive baseline, confirming that deliberation without compartmentalization is vulnerable to coordination.
Generalization is tested against CorruptRAG-AS, which exploits LLM update bias via template-based correction framing rather than learned pipeline robustness. On SciFact, vanilla RAG ASR matches Confundo's 62.0%, showing update-bias framing is as effective as LLM-optimized poisoning against undefended systems. Cordon-MAS with an enhanced Auditor (factual plausibility checking against parametric knowledge, plus uniform-agreement detection) reduces ASR to 26.0%—the highest residual ASR observed in any evaluation, and a meaningful vulnerability when all retrieved documents are mutually consistent poison. The enhanced Auditor cuts certification of poison claims from 64.1% to 31.8%, validating defense-in-depth through an audit signal orthogonal to consistency.
An n=100 validation on SciFact and NQ reveals substantial seed-dependent variance: Cordon-MAS SciFact ASR ranges from 2.0% (seed 42, base Auditor) to 26.5% (seed 100, enhanced Auditor), with the base Auditor at 46.9% under seed 100. The authors identify the n=100 estimate as more reliable for SciFact while retaining n=50 results for baseline comparability. LLM-judge reliability is validated by human annotation (98% agreement, Cohen's κ=0.96), with the judge's conservative bias implying reported ASR values are upper bounds.
Limitations and open questions
The paper is explicit about several constraints. Multi-document consistency collusion is a fundamental, information-theoretic limitation: when the attacker controls the entire visible evidence set, no within-corpus signal can distinguish collusion from consensus; mitigation requires external trust anchors (provenance metadata, external knowledge bases, document signatures), which the architecture could accommodate as Auditor plugins but which remain unimplemented. Clean utility degrades on multi-hop queries (HotpotQA, 40% answerability) because the Extractor processes documents independently, leaving cross-document inference chains invisible to audit. Inference overhead is 2.2× latency and 2.8× cost versus vanilla RAG (3–4 LLM calls per query). The homogeneous backend assumption—all agents sharing DeepSeek-Chat—leaves open whether audit effectiveness depends on shared representational biases between Extractor and Auditor; heterogeneous-backend evaluation is an open question. Finally, evaluation covers only factual-manipulation attacks (Confundo's opinion manipulation and hallucination induction are unevaluated), uses a single retriever, and relies on 50-query samples with per-dataset CIs of 5–15 percentage points; the extraction-yield analysis also notes the reported ASR is a lower bound if extraction quality improves, since unextracted poison is invisible to measurement.
Conclusion
Cordon-MAS reframes RAG knowledge poisoning from a detection problem to an information-flow control problem. Its three supporting lines of evidence are the empirically demonstrated monitoring-control gap (prompt-based detection reduces but cannot eliminate endorsement), the informal attention-contamination analysis explaining why prompting is structurally leaky, and a security-utility Pareto frontier on which compartmentalization dominates. The headline result—a 92.4% relative ASR reduction (2.1% vs. 27.5%) with cross-backend invariance—is qualified by a well-characterized residual vulnerability to coordinated multi-source poisoning (26.5–26.0% ASR under the strongest adaptive and update-bias attacks), which the authors identify as the boundary where within-corpus verification must be supplemented by external trust anchors.