MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval (2512.16962v1)

Abstract: LLM agents increasingly rely on long-term memory and Retrieval-Augmented Generation (RAG) to persist experiences and refine future performance. While this experience learning capability enhances agentic autonomy, it introduces a critical, unexplored attack surface, i.e., the trust boundary between an agent's reasoning core and its own past. In this paper, we introduce MemoryGraft. It is a novel indirect injection attack that compromises agent behavior not through immediate jailbreaks, but by implanting malicious successful experiences into the agent's long-term memory. Unlike traditional prompt injections that are transient, or standard RAG poisoning that targets factual knowledge, MemoryGraft exploits the agent's semantic imitation heuristic which is the tendency to replicate patterns from retrieved successful tasks. We demonstrate that an attacker who can supply benign ingestion-level artifacts that the agent reads during execution can induce it to construct a poisoned RAG store where a small set of malicious procedure templates is persisted alongside benign experiences. When the agent later encounters semantically similar tasks, union retrieval over lexical and embedding similarity reliably surfaces these grafted memories, and the agent adopts the embedded unsafe patterns, leading to persistent behavioral drift across sessions. We validate MemoryGraft on MetaGPT's DataInterpreter agent with GPT-4o and find that a small number of poisoned records can account for a large fraction of retrieved experiences on benign workloads, turning experience-based self-improvement into a vector for stealthy and durable compromise. To facilitate reproducibility and future research, our code and evaluation data are available at https://github.com/Jacobhhy/Agent-Memory-Poisoning.

• The paper demonstrates that poisoned experience retrieval can persistently compromise LLM agents by injecting malicious behavior into their memory stores.
• It experimentally shows that a small number of poisoned entries—up to 47.9% of retrievals—can dominate outcomes and induce lasting behavioral drift.
• It proposes defense mechanisms like Cryptographic Provenance Attestation and safety-aware reranking to mitigate vulnerabilities in memory-based retrieval systems.

Persistent Compromise of LLM Agents via Experience Retrieval Poisoning

Introduction

The paper "MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval" (2512.16962) rigorously examines the threat posed by memory poisoning attacks targeting the durable experience stores of agentic LLM systems. As modern LLM agents increasingly employ Retrieval-Augmented Generation (RAG) architectures to persist experiential traces and leverage them for future reasoning, their emergent memory modules represent a fundamentally new—and largely unprotected—attack surface. MemoryGraft exploits the semantic imitation heuristic: the agent’s propensity to treat retrieved prior success traces as procedural ground truth. The attack is distinguished from traditional prompt injection and RAG poisoning because it operates in the long-term memory plane, induces persistent behavioral drift, and is entirely trigger-free.

Attack Methodology

MemoryGraft leverages standard agent ingestion channels to implant malicious yet plausible prior success examples into the memory pipeline. A sophisticated adversary crafts benign-appearing documentation (e.g., README files) embedding executable payloads and hidden unsafe patterns, subsequently causing the agent to ingest and persist attacker-constructed experience templates. Crucially, these entries encode unsafe procedures (e.g., skipping validation, running remote scripts), camouflaged as "validated/safe" best practices.

Figure 1: MemoryGraft attack lifecycle and the persistence of compromised behaviors across sessions due to poisoned procedural templates in the RAG store.

Upon subsequent clean queries, the agent’s semantic and lexical retrieval unions (BM25 + FAISS-based similarity) consistently surface these grafted experiences, resulting in imitation of unsafe behaviors. The system’s lack of explicit provenance tracking or adversarial filtering allows the compromise to propagate across sessions, persisting until manual memory sanitization.

Experimental Results

Empirical validation was conducted on MetaGPT's DataInterpreter agent, using GPT-4o as the reasoning core. A seed set of 110 records (100 benign and 10 poisoned) was injected via a Markdown payload containing runnable code to build the memory store. Evaluation against 12 realistic, semantically varied queries revealed the following:

• Out of 48 total retrievals, 23 were poisoned entries (Poisoned Retrieval Proportion $= 47.9\%$), demonstrating substantial infiltration by the malicious seeds, despite comprising <10% of the memory dataset.
• Poisoned records surfaced for both narrowly aligned and semantically distant queries, confirming that the dual retrieval mechanism (BM25 + embeddings) dramatically expands the attack’s basin of attraction.
• Behavioral drift was observed across diverse workloads—data cleaning, schema handling, anonymization, and reporting—underscoring the attack's robustness and lack of trigger dependence.

These findings make clear that the persistence and efficacy of MemoryGraft are not bounded by superficial query overlap; poisoned experiences can dominate retrieval outcomes due to their strategic semantic proximity to ubiquitous analytic workflows.

Mechanism Analysis and Defense Proposals

The attack's durability and stealth stem from two intertwined mechanisms:

• Benign-Poisoned Balance: By maintaining $n_{\mathrm{poison}} \ll n_{\mathrm{benign}}$, the poisoned store statistically appears healthy, yet the tailored proximity of poisoned records ensures frequent retrieval in common user scenarios.
• Cross-Session Persistence: Serialization of the compromised memory store means that all future agent invocations—including those by other users—are subject to the same contaminated retrievals.

Proposed mitigations include:

• Cryptographic Provenance Attestation (CPA): Mandating digital signatures for validated agent experiences, where only self-generated and cryptographically attested records are accepted as trusted retrievals.
• Constitutional Consistency Reranking: Introducing a safety-aware reranking step, which penalizes retrieved traces showing divergence from the agent’s safety constitution, using automated risk scoring and entailment checks to suppress unsafe patterns post-retrieval.

Both schemes require agent architectural modifications, but are theoretically sound in blocking external artifacts from surreptitiously contaminating experience memories and biasing agentic behavior.

Theoretical and Practical Implications

The MemoryGraft attack fundamentally recasts the RAG and agent memory substrate as a critical security liability. By demonstrating persistent, session-independent behavioral drift of LLM agents via memory poisoning, this work challenges the presupposition of trust between the agent reasoning core and its historic experience bank. The agent's reliance on semantic similarity, and absence of provenance or risk-aware vetting, is shown to permit stealthy, durable compromise that remains invisible to existing prompt- and RAG-based defenses.

Practically, this implies that deployment of agentic LLMs in critical enterprise or compliance-sensitive workflows is unsafe without robust memory vetting protocols. Theoretically, the findings call for a re-evaluation of agent learning paradigms—experience-based self-improvement must be constrained by mechanisms that ensure the integrity and authenticity of the experiences themselves. System designs lacking these controls are vulnerable to long-term, indirect adversarial steering, regardless of prompt-level or context window sanitization.

Limitations and Future Directions

The presented evaluation, based on a handcrafted seed set and a single agent archetype, leaves questions regarding generalization to more heterogeneous environments and larger, multi-agent workflows. Further research is warranted in black-box threat models, longitudinal drift characterization, and development of standardized memory poisoning benchmarks. The spread of contaminated experiences in collaborative or multi-agent scenarios merits deeper investigation, particularly as model architectures trend toward shared multi-agent memory pools.

Conclusion

MemoryGraft presents a technically rigorous demonstration of the persistent compromise of LLM agent behavior via poisoned experience retrieval. By exploiting the agent imitation heuristic and the lack of memory provenance, an adversary can induce durable behavioral drift utilizing nothing more than legitimate documentation ingestion channels. Nearly half of all retrieved experiences following attack were poisoned templates, a strong claim supported by rigorous empirical analysis. Future agent architectures must treat the experience memory pipeline as a first-class security boundary, incorporating cryptographic attestation and risk-aware reranking to mitigate stealthy compromise. This research compels the community to acknowledge memory poisoning as a central vulnerability in data-driven agent autonomy.