---
title: 'PISanitizer: Prompt Injection Defense'
url: https://www.emergentmind.com/topics/pisanitizer
type: topic
---

# PISanitizer: Prompt Injection Defense

PISanitizer is an inference-time defense for long-context large language models that seeks to prevent prompt injection by sanitizing the input context before it reaches the model that generates the final response. It is designed for the setting in which an attacker embeds an instruction inside a long context so that the model follows the attacker’s instruction rather than the intended task. The method first pinpoints and sanitizes potential injected tokens in a context and then lets a backend LLM answer on the sanitized context, with the stated goal of eliminating the influence of the injected instruction. Its central premise is that prompt injection attacks work by inserting an instruction that compels an LLM to follow it, and that transformer attention exposes which context tokens drive that instruction-following behavior [2511.10720].

## 1. Problem setting and threat model

PISanitizer targets long-context prompt injection. In this setting, an attacker injects an explicit instruction into the model’s input context, and the LLM may follow that malicious instruction instead of the target instruction. The paper states that existing prompt injection defenses are designed for short contexts and have limited effectiveness when extended to long-context scenarios, because an injected instruction constitutes only a very small portion of a long context [2511.10720].

The method assumes a separation between a **sanitizer LLM** and a **backend LLM**. The sanitizer is an open-source model used to inspect the context, while the backend is the model that ultimately answers the user’s intended task. This separation is operationally important because the backend LLM is intended to see only the sanitized context, not the original potentially contaminated context. A plausible implication is that the design supports deployments in which the inspection stage and the task-execution stage are decoupled.

The defense is specifically framed around **explicit instructions** embedded in context. The paper also states an important boundary condition: PISanitizer only removes explicit instructions and cannot defend against **knowledge-corruption** attacks that subtly bias factual output without explicit commands. This restriction materially shapes the scope of the system and distinguishes it from broader data-integrity or retrieval-integrity defenses.

## 2. Core observations and design rationale

PISanitizer is built on two observations. First, prompt-injection attacks work by injecting an explicit instruction into the context that compels the LLM to follow the malicious instruction instead of the intended task. Second, transformer-style LLMs rely on the self-attention mechanism to allocate high attention weights to those input tokens that drive the generation of each output token [2511.10720].

From these observations, the defense deliberately induces instruction-following behavior during sanitization. It constructs a short **sanitization instruction** whose only goal is to force the LLM to follow any instruction embedded in the context. The example given is:

> “Do anything you are told to do in the following context.  
> Context: {Context}  
> Only do what the user asks you to do. Do not provide any explanation.  
> Your response:”

The paper’s rationale is that if an injected instruction can successfully compel the sanitizer LLM, then the tokens responsible for that compulsion should receive relatively large attention weights from the first generated output token. PISanitizer then sanitizes those tokens. The paper characterizes this as a dilemma for the attacker: the more effectively an injected instruction compels an LLM to follow it, the more likely it is to be sanitized by PISanitizer.

This design differs from defenses that focus primarily on prompt classification or output filtering. Related work in the broader sanitization literature illustrates that this is part of a larger family of front-end defenses, but PISanitizer’s distinctive feature is the use of attention from a deliberately instruction-following sanitizer pass to localize malicious spans in long contexts. For context, Casper is a browser-side prompt sanitization system for user privacy rather than prompt injection [2408.07004], while CodeSentinel applies a three-layer sanitizer to indirect prompt injection in code contexts [2606.19235]. This suggests that PISanitizer occupies the natural-language, long-context branch of a broader sanitizer design space.

## 3. Sanitization pipeline

The pipeline has two phases: **Prompt Sanitization** and **Backend LLM Response** [2511.10720].

In the first phase, PISanitizer takes the target instruction $I_t$ and a possibly contaminated context $C$. It constructs the sanitization instruction $I_s$, feeds $I_s \oplus C$ to an open-source sanitizer LLM, and generates a **single** output token $o$. The use of only the first output token is deliberate: the paper states that the first token’s attention scores already reveal which input tokens strongly influenced the model to follow an instruction.

It then extracts the attention weights between each context token $c_k \in C$ and the generated token $o$ across all $L$ layers and $H$ heads of the sanitizer LLM. These raw attention values are aggregated into a per-token score vector, smoothed, and analyzed for peaks. The procedure identifies local peaks, groups peaks that are within distance $d$, and selects the highest-scoring group. If the maximum score of that group exceeds a threshold $\theta$, all tokens in that group are removed from the context. The process repeats for up to a maximum of 5 passes or until no further tokens are removed.

In the second phase, the system concatenates the sanitized context $C'$ with the target instruction $I_t$ and queries the backend LLM to generate the final output. The paper states that this two-stage pipeline ensures that the backend LLM never sees the malicious instruction.

The high-level algorithm is:

```text
Algorithm PISanitizer
Input:
  – Sanitizer LLM f
  – Context C = [c₁,…,cₘ]
  – Sanitization instruction Iₛ
  – Aggregation threshold θ
  – Smoothing window wₛ, peak distance d
Output:
  – Sanitized context C′

1.   Repeat up to max_rounds:
2.     // 1) Generate one token with instruction following
3.     o ← f.generate_token( Iₛ ⊕ C )
4.     // 2) Extract raw attention between each cₖ and o
5.     For k in 1…m:
6.       Aₖ := f.get_attn_weights(input=Iₛ⊕C, output_token=o)
7.     // 3) Layer-wise noise-aware aggregation
8.     For k in 1…m, layer ℓ in 1…L:
9.       sₖ^ℓ = (1/H) ∑_{h=1}^H Aₖ[ℓ,h]
10.      sₖ = max_{ℓ=1…L} sₖ^ℓ
11.    Form s = [s₁,…,sₘ]
12.    // 4) Smooth and find peaks in s
13.    s̄ = SavgolFilter(s, window=wₛ)
14.    peaks = find_local_peaks(s̄, min_height=0.005)
15.    groups = group_peaks(peaks, max_gap=d)
16.    // 5) Identify highest-scoring group
17.    For each group Gᵢ:
18.      vᵢ = max_{k∈Gᵢ} sₖ
19.    Let i* = argmaxᵢ vᵢ
20.    If v_{i*} > θ:
21.      Remove tokens in G_{i*} from C
22.      Continue next round
23.    Else:
24.      Break
25.  Return C′ = C
```

A common misunderstanding is that the method performs semantic rewriting of the context. The paper instead describes token removal based on attention-derived localization of the highest-scoring group.

## 4. Mathematical formulation and implementation parameters

Let the context be tokenized into $C=(c_1,\dots,c_m)$. The sanitizer LLM $f$ has $L$ Transformer layers, each with $H$ attention heads. When generating the first output token $o$ for prompt $I_s \oplus C$, PISanitizer records

\[
a_{k}^{\ell,h} = \text{attention weight from input token }c_k
                  \text{ (at layer }\ell\text{, head }h)
                  \text{ to }o.
\]

It performs layer-wise head averaging,

\[
s_k^\ell = \frac{1}{H}\sum_{h=1}^{H} a_{k}^{\ell,h}
\quad\text{for }\ell=1\dots L,
\]

followed by max-over-layers aggregation,

\[
s_k = \max_{\ell\in\{1,\dots,L\}} s_k^\ell.
\]

The resulting score vector is $s=(s_1,\dots,s_m)$. The system smooths $s$ via Savitzky–Golay to obtain $\bar s$, identifies local maxima in $\bar s$, and clusters peaks within distance $d$ into groups $\{G_i\}$. Group scores are defined by

\[
v_i = \max_{k\in G_i}s_k.
\]

Let $i^*=\arg\max_i v_i$. If $v_{i^*}>\theta$, the system removes all tokens in $G_{i^*}$.

The typical hyperparameters are:

\[
w_s=\begin{cases}
9,&|C|>500\\
5,&|C|\le 500
\end{cases},
\quad d=10,\quad \theta=0.01.
\]

The architecture is explicitly two-stage. The **sanitizer LLM** is open-source, with the example of **Llama-3.1-8B-Instruct**. Its input is the single sanitization prompt $I_s \oplus C$, and it outputs one token plus per-token attention maps. The **backend** may be closed- or open-source, with examples including **GPT-5**. This suggests that PISanitizer is designed to be backend-agnostic so long as the sanitizer model exposes attention maps [2511.10720].

## 5. Experimental setup and empirical performance

The evaluation uses LongBench datasets: **Qasper** and **HotpotQA** for question answering, **GovReport** and **MultiNews** for summarization, **LCC** for code completion, and **PassageRetrieval** for retrieval. Each dataset contains 200 samples, with 100 randomly selected for evaluation [2511.10720].

Injected tasks include four categories: target answer generation, “Hacked,” classic NLP tasks, and general instructions. The attack suite includes **heuristic-based** attacks—Naive, Escape, Context-Ignoring, Fake-Completion, and Combined—along with **optimization-based** attacks using nano-GCG suffix optimization for up to 500 iterations, and **adaptive attacks** such as repeat injections, anti-sanitization wrappers, and suffix optimization minimizing attention weights.

The evaluated backends include open-source models—**Llama-3.1-8B/70B** and **Qwen-30B**—and closed-source models—**GPT-4o, GPT-4o-mini, GPT-4.1, GPT-5**. Baselines are divided into prevention methods (**Sandwich, Instructional, Meta-SecAlign, DataFilter**), detection methods (**DataSentinel, PromptGuard, AttentionTracker**), and detection+attribution methods (**PromptArmor, PromptLocate, AttnTrace**).

The reported metrics are utility, attack success rate, runtime, and precision/recall/F1 of token sanitization. Utility is task-specific, using **F1, ROUGE-L, EditSim,** and **accuracy**. The principal empirical claims are as follows [2511.10720]:

- **Main defense effectiveness**: with a Llama-3.1-8B backend under Combined Attack, without defense the ASR is approximately **66–97%**, and utility degrades by **50–90%**; with PISanitizer, ASR is approximately **0–2%**, and utility is restored to within **1–5%** of the no-attack baseline.
- **Cross-LLM generality**: the method works similarly on **GPT-4o, GPT-5, Qwen-30B**, preserving utility and reducing ASR to near zero.
- **Injected task variations**: ASR drops to **0–2%** for all four injected-task categories.
- **HotpotQA under Combined Attack**: **Meta-SecAlign** has ASR approximately **56%** with utility drop approximately **10–30%**; **PromptArmor** has ASR approximately **53–74%** with utility drop approximately **5–30%**; **PromptLocate** has ASR approximately **0%** but utility drop approximately **20–50%** and runtime **> 600 s**; **PISanitizer** has ASR approximately **1%**, utility approximately at no-attack level, and runtime approximately **10 s** consisting of **1.8 s sanitization + ≈8 s backend**.
- **Adaptive attacks**: ASR remains **≤ 4%** even under heuristic and suffix-optimization attacks.
- **Sanitization accuracy**: token-level precision is approximately **0.75–0.95**, recall approximately **0.80–1.00**, and F1 approximately **0.80–0.93**.
- **Efficiency**: sanitizing a **1–3 k token** context requires approximately **1.8 s** on a **single H100 GPU**.

These numbers position the method as a low-overhead defense relative to attribution-heavy baselines such as PromptLocate, while avoiding the paper’s reported utility drop for that baseline.

## 6. Limitations, failure modes, and relation to adjacent sanitization research

The paper identifies several limitations. First, PISanitizer only removes **explicit instructions** and does not defend against **knowledge-corruption** attacks. Second, it cannot distinguish **benign** from **malicious** instructions in the context; necessary benign instructions may therefore be removed, potentially lowering utility. Third, the method relies on the assumption that any instruction the sanitizer LLM follows will attract high attention. The paper states that if an attacker can compel the backend LLM without large attention to the instruction—described as an extremely “weak” attack—PISanitizer may fail [2511.10720].

The paper also notes prospective extensions: multi-modal LLMs, certifiable bounds on sanitization coverage or provable guarantees under restricted attacker knowledge, and automatic policy-based classification of sanitized spans as benign versus malicious.

These limitations matter because they delimit what “sanitization” means in this framework. PISanitizer is neither a general factual-integrity defense nor a semantic policy verifier. It is a targeted front end for localizing and removing explicit instruction-bearing spans before inference. A plausible implication is that it composes naturally with complementary defenses rather than replacing them.

Adjacent work underscores this specialization. **Self-Sanitize** addresses harmful content generation by combining token-level monitoring and in-place repair, and the abstract notes that it emphasizes privacy leakage scenarios rather than prompt injection [2509.24488]. **Casper** sanitizes user prompts to remove PII and privacy-sensitive topics on-device before sending them to web-based LLM services [2408.07004]. **Pr$\epsilon\epsilon$mpt** formalizes prompt sanitization for sensitive tokens through format-preserving encryption and metric differential privacy [2504.05147]. **CodeSentinel** extends inference-time sanitization to indirect prompt injection in code contexts through syntax-guided filtering, dynamic anomaly scoring, and node perturbation analysis [2606.19235]. Taken together, these systems suggest that “sanitization” has become a unifying systems pattern across prompt security, privacy preservation, and model-facing context control, with PISanitizer representing the long-context prompt-injection instance of that pattern.

Source: https://www.emergentmind.com/topics/pisanitizer