GIF: Locally Sound Geometric Information Flow Control for LLMs

Abstract: LLMs increasingly mediate interactions between sensitive data, untrusted inputs, and privileged actions in agentic systems, creating security and privacy risks. These range from prompt injections that manipulate downstream tool use to leakage of confidential information through model outputs. Recent Information Flow Control (IFC)-based defenses show promise but lack a principled semantic foundation for reasoning about information flow through the model itself. Since any input token may influence any output token in an autoregressive LLM, existing approaches suffer from severe taint explosion. We present Geometric Information Flow (GIF), a semantic framework for tracking information flow from input tokens to outputs. GIF uses the LLM Jacobian and local output geometry to upper-bound the Shannon mutual information between perturbed input spans and model outputs, yielding a scalable measure computable on large models via automatic differentiation and low-rank approximation. Unlike attention-based or correlational attribution heuristics, GIF satisfies local geometric soundness, and we provide a fully mechanized Lean 4 proof that it upper-bounds the true information flow induced by a given prompt under local regularity assumptions. We evaluate GIF on integrity and confidentiality tasks across multiple prompt-injection and privacy-leakage benchmarks. GIF achieves near-perfect recall even without a downstream declassifier, outperforming attention-based baselines. Combined with lightweight LLM-based declassifiers, it matches or exceeds the F1 of direct LLM-as-judge baselines such as GPT-5.5 xhigh reasoning while using up to 81x lower token cost. GIF flows detected with small surrogate models transfer to larger state-of-the-art models and other model families, even when the surrogate is up to 200x smaller, suggesting black-box deployment without gradient access.

• The paper introduces GIF, a locally sound information-flow measure based on mutual information, Fisher pullback geometry, and a mechanized theorem showing true local flow is bounded by estimated capacity up to o(τ).
• The method uses Hutch++ trace estimation for scalable influence ranking, achieving Spearman correlation up to 0.988 and reducing attribution costs by about 9.3× versus GPT-5.5 while improving average AUROC by 11.2%.
• GIF improves prompt-injection and privacy-leakage detection across three benchmarks, matches GPT-5.5-guided declassification with Qwen 3 8B at over 81× lower cost, and can certify near-zero flow to reduce overtaint.

Motivation and problem statement

Agentic LLM systems combine user intent, untrusted inputs, private context, and privileged actions inside opaque autoregressive model invocations. This produces two dual failure modes: integrity violations, where attacker-controlled content (e.g., indirect prompt injections) influences trusted actions such as tool calls or code edits, and confidentiality violations, where private context leaks through generated outputs. Existing information-flow control (IFC) defenses for agents attach coarse labels (trusted/untrusted, public/confidential) but cannot propagate them through the model: since any input token may weakly influence any output token, conservative propagation causes severe overtaint, while ad hoc filters and output checks trade overtaint for blind spots. The paper's central claim is that what is missing is a quantitative information-flow semantics for the model invocation itself — a measure of how much local information about an input span is observable in a downstream output.

The GIF framework

Geometric Information Flow (GIF) fixes a base prompt $x$, an input span $T$, and an output position $t^\star$, and defines flow as the Shannon mutual information between a Gaussian perturbation of the span's embeddings and the sampled next token. Because exact mutual information for the true autoregressive channel is intractable, the authors construct a linear Gaussian surrogate channel whose per-perturbation KL divergence matches the true channel's local KL geometry. The key object is the Fisher pullback

$$M_T(x) = J_T(x)^\top W^\top F_{\mathrm{sm}(p)}\, W J_T(x),$$

where $J_T$ is the span-to-hidden-state Jacobian, $W$ the LM head, and $F_{\mathrm{sm}(p)} = \mathrm{diag}(p) - pp^\top$ the softmax Fisher metric. Two scalar summaries are read off its spectrum: the influence $\mathrm{Inf}_T = \mathrm{tr}\, M_T$ and the capacity $$\mathrm{Cap}_T(\tau) = \tfrac12 \log\det(I + \tau M_T)$$, which is exactly the surrogate channel's mutual information. The operational measure is $\mathrm{GIF}_T(\tau;x) := \mathrm{Cap}_T(\tau;x)$, with influence as a first-order proxy satisfying $T$0.

Mechanized soundness

The theoretical contribution is a local soundness theorem: under a local-regularity assumption (a quadratic remainder bound on the hidden-state map around the base prompt), the true flow satisfies

$T$1

The proof proceeds in three moves: a variational reduction replacing the intractable output marginal by the base distribution (which can only inflate leakage), Gaussian averaging of the uniform second-order faithfulness bound, and capacity tightening via the $T$2 gap between influence and capacity. Notably, every mathematical claim is formalized and machine-checked in Lean 4, including the reduction from Hutch++'s trace guarantee to a probabilistic soundness corollary. The soundness guarantee is explicitly local — it holds to leading order in the perturbation scale under regularity assumptions, not globally over arbitrary prompt rewrites — and the Lean badges certify mathematical claims, not deployed-model behavior.

Scalable estimation

Materializing $T$3 would require one backward pass per hidden dimension (up to 7168 for large open-weight models). Instead, the system estimates the influence trace with Hutch++, reducing computation to a small number of matrix-vector products computed via forward- and reverse-mode automatic differentiation at the fixed prompt. With $T$4 probes, the estimate is within relative error $T$5 with probability $T$6, yielding a probabilistic version of the soundness bound. Ablations show that ranking quality saturates early: Spearman correlation against exact influence is already 0.957 at sketch rank 1 versus 0.988 at rank 64, so low-rank sketches suffice when GIF is used to rank evidence rather than calibrate magnitudes.

System design

The implementation has three components: heuristic two-point lattice labeling (Low/High for both confidentiality and integrity), a flow-tracking component that computes GIF scores from sink tokens back to prefix sources during generation, and a declassifier. Two declassifier designs are evaluated: an oracle threshold declassifier using Policy Source Precision at cutoff $T$7 ($T$8, evaluated via AUROC/AUPRC sweeps to avoid calibration assumptions), and an LLM-as-declassifier that receives only the top-$T$9 highest-influence source spans embedded in minimal context. The authors state plainly that labeling heuristics and formal declassifier semantics are left to future work; GIF's contribution is the flow signal itself.

Evaluation

Experiments cover three benchmarks (AgentDojo and MSB for prompt injection; AgentDAM for privacy leakage), six open-weight models (Qwen 3 8B/14B/32B, Gemma 4 31B, GPT-OSS 20B/120B), and baselines including attention-based RTBAS, a single-logit gradient variant (GIF⁻), and GPT-5.5 xhigh as an attribution judge. Key results:

Comparison	Result
GIF vs. GPT-5.5 xhigh attribution	+11.2% AUROC, +12.0% AUPRC on average, at ~9.3× lower cost ($t^\star$0147.90)
GIF vs. RTBAS	Substantially higher AUROC/AUPRC across all datasets; e.g., average AgentDojo AUROC 0.91 vs. 0.84 at @50
GIF-guided Qwen 3 8B declassifier	Matches GPT-5.5 xhigh on AgentDojo and MSB at >81× lower cost
Token savings vs. full-trajectory judging	0.27–0.37× tokens on AgentDojo/MSB (63–73% reduction); quality ratio 0.97–1.22×

Two caveats are conceded directly. On Gemma 4 31B, GPT-5.5 achieves higher AUPRC, attributed to sparse successful injections and long continuous injection spans favorable to the judge. On AgentDAM, token savings shrink to 0.76–0.81× because the browser-based agent keeps little conversation history, leaving little context for GIF to remove.

Robustness results support black-box deployment: attributing trajectories from Qwen 3 32B or GPT-OSS 120B with a fixed Qwen 3 0.6B surrogate changes downstream detection metrics by at most 0.9 points, even though the surrogate is up to 200× smaller and cross-family. The cutoff sweep shows AUROC and AUPRC reach 95% of their total gain by $t^\star$1 and $t^\star$2 respectively, indicating headline results are not artifacts of a particular cutoff.

Case studies illustrate qualitative behavior GIF enables: in a partial-success attack where the benchmark oracle records failure, GIF still attributes 31% of the recipient address's flow to the injected instruction while RTBAS misses it entirely; conversely, in a zero-outflow case, GIF assigns 0 bits to a private address that never surfaces in the output, whereas last-layer attention still taints it at 9%. The ability to certify near-zero flow is what allows GIF to avoid the overtaint that plagues conservative label propagation.

Limitations and open questions

Several limitations are explicit. Soundness is local and asymptotic: it holds under a quadratic-remainder regularity assumption and only up to $t^\star$3 corrections, so GIF does not certify global noninterference, which the authors argue is unattainable for autoregressive models anyway. The Hutch++ guarantee is adopted as a hypothesis rather than re-proven in Lean. Labeling remains heuristic, and no formal declassifier semantics for agentic systems is provided. The evaluation uses open-weight models because GIF requires internals; transfer to closed models rests on the surrogate result, which is demonstrated empirically rather than theoretically. Finally, the exact-versus-approximate comparison is restricted to a small sink-token sample, since full exact computation over trajectories is prohibitively expensive.

Conclusion

GIF provides a quantitative, locally sound semantics for information flow through LLM invocations, grounded in mutual information over a mechanized Fisher-geometry surrogate and made practical by randomized trace estimation. Its empirical results — near-perfect recall without declassification, consistent superiority over attention-based attribution, frontier-judge-matching accuracy at up to 81× lower cost, and cross-scale surrogate transfer — establish it as a viable flow-tracking substrate for agentic IFC. The framework leaves open how to build principled labeling policies and formal declassifier semantics on top of the measured flows.