DeLeaker: AI Leakage Mitigation

• DeLeaker is a framework that diagnoses and mitigates semantic, memory, and privacy leakage in AI models through systematic, inference-time interventions.
• It leverages dynamic reweighting of transformer attention in text-to-image diffusion, IPG-based memory auditing, and attribution hardening against membership inference.
• Empirical benchmarks show significant leakage reduction—up to 95% in some settings—while maintaining high model fidelity and utility.

DeLeaker denotes a class of methods and systematizations for diagnosing and mitigating information leakage—semantic, privacy, or memory-based—in modern AI models. In contemporary usage, it references (1) dynamic inference-time reweighting to mitigate semantic leakage in text-to-image (T2I) diffusion models and (2) the systematic detection and reduction of memory leaks at the fact level in multimodal agent memory and (3) privacy hardening for model explanations against membership inference. These usages converge on the objective of minimizing unintended information transfer across representations, entities, or data modalities.

1. Formal Definition and Taxonomies

In text-to-image models, DeLeaker is defined as a lightweight, optimization-free, inference-time approach that directly suppresses semantic leakage—unintended visual feature transfer between entities—by reweighting attention maps throughout diffusion (Ventura et al., 16 Oct 2025). Formally, if $e_i, e_j$ are distinct entities in a prompt and $\mathcal{E}_i^{\text{img}}$ denotes the set of generated image positions governed by $e_i$, leakage is present if attention scores or pixel values in $\mathcal{E}_j^{\text{img}}$ strongly reflect features characteristic of $e_i$ rather than $e_j$. The mathematical machinery centers on modulating transformer attention at each diffusion step by thresholding and modifying inter-entity interactions.

For multimodal memory deletion, DeLeaker refers to a system that audits and mitigates recoverability of deleted facts in AI agent memory, especially concerning the persistence of information through correlated text or retained images (Wang et al., 29 Jun 2026). This is formalized using the Information Provenance Graph (IPG), a directed acyclic graph $G_f = (V_f, E_f, \delta)$ per fact $f$, with nodes for each artifact (text entries, embeddings, image files, activations) and edges indicating derivation. Deletion affordance is classified as Addressable, Linked, or Persistent, revealing distinct leakage channels.

Within the context of model explanations, DeLeaker (as codified in DeepLeak) audits and mitigates membership inference leakage in post-hoc explanation vectors through evaluation of Membership Leakage Score (MLS) and a set of attribution hardening strategies (Hmida et al., 6 Jan 2026).

2. Core Methodologies

2.1 Dynamic Inference-Time Attention Reweighting (T2I)

DeLeaker intercepts the pre-softmax attention scores at each transformer block and diffusion step, applying the following modification:

$$\text{Attn}'_{qk} = \begin{cases} -\infty, & q\in\mathcal{E}_i^{\mathrm{img}}, k\in\mathcal{E}_j^{\mathrm{img}}, (q,k)\in H_{ij}^{\mathrm{img-img}} \ -\infty, & q\in\mathcal{E}_i^{\mathrm{img}}, k\in\mathcal{E}_j^{\mathrm{txt}}, i\neq j \ \alpha\,\text{Attn}_{qk}, & q\in\mathcal{E}_i^{\mathrm{img}}, k\in\mathcal{E}_i^{\mathrm{txt}} \ \text{Attn}_{qk}, & \text{otherwise} \end{cases}$$

where $\mathcal{E}_i^{\mathrm{img}}$ and $\mathcal{E}_i^{\text{img}}$0 are entity-specific masks, $\mathcal{E}_i^{\text{img}}$1 indexes strong cross-entity attention, and $\mathcal{E}_i^{\text{img}}$2 strengthens within-entity consistency. This disables cross-entity leakage while reinforcing compositional fidelity (Ventura et al., 16 Oct 2025).

2.2 Memory Leak Diagnosis and Deletion (Multimodal Agents)

• IPG Construction: Systematically tracks all concrete and derived memory representations of each fact $\mathcal{E}_i^{\text{img}}$3, marking their deletion affordance (Addressable, Linked, Persistent).
• Deletion Cascade Measurement: Applies successively stronger deletion policies, quantifying leakage via direct, text-mediated, and image-mediated recovery metrics $\mathcal{E}_i^{\text{img}}$4, $\mathcal{E}_i^{\text{img}}$5, $\mathcal{E}_i^{\text{img}}$6.
• Content-Aware Semantic Deletion: After initial “forget” operations, a VLM is used to audit remaining assets, deleting images $\mathcal{E}_i^{\text{img}}$7 with high semantic similarity to the forgotten fact ($\mathcal{E}_i^{\text{img}}$8), empirically reducing $\mathcal{E}_i^{\text{img}}$9 by 6$e_i$0 (Wang et al., 29 Jun 2026).

2.3 Explanation Privacy Hardening (Membership Leakage)

• Explanation-aware Membership-Inference Attack: Trains a binary classifier to distinguish training members from non-members using attribution vectors $e_i$1, computing MLS at fixed FPR.
• Mitigation Pipeline: Sequentially applies attribution clipping, low-signal masking, and calibrated Gaussian noise:
  • $e_i$2
  • $e_i$3 if $e_i$4, else $e_i$5
  • $e_i$6, $e_i$7.
  • Order and parameter tuning are bi-objective: minimize MLS s.t. $e_i$8 (sensitivity loss) remains low (Hmida et al., 6 Jan 2026).

3. Benchmarks and Quantitative Results

Setting	Leakage (Baseline)	Leakage (DeLeaker)	Fidelity Loss (Utility)
T2I Semantic Leakage (Ventura et al., 16 Oct 2025)	See below	55.8% mitigation	LPIPS 0.22, KID 0.00
Multimodal Agent $e_i$9 (Wang et al., 29 Jun 2026)	12.0%	2.0% (semantic cut)	~21% images removed
Explainer MLS, e.g. SmoothGrad (Hmida et al., 6 Jan 2026)	10.38%	0.84%	–22.4% sensitivity

In T2I generation, DeLeaker delivers major reduction in visual leakage without degrading prompt-image alignment, as measured by VQAScore, LPIPS, and KID. In multimodal memory, content-aware semantic deletion reduces recoverability from 12.0% to 2.0%, with category-uniform protection. For post-hoc explainers, combination of clipping, masking, and calibrated noise yields up to 95% reduction in membership leakage at ≤3.3% utility cost.

4. Auditing, Evaluation, and Metrics

For T2I leakage, system-level evaluation is grounded in the SLIM (Semantic Leakage in IMages) benchmark, featuring 1,130 human-verified prompt–seed–image triples across fine-grained, multi-entity scenarios.

For memory leaks, the MemLeak protocol combines synthetic and real-image experiments under controlled fact injection, deletion, and VLM-based probing (Wang et al., 29 Jun 2026). Key metrics:

$\mathcal{E}_j^{\text{img}}$0

$\mathcal{E}_j^{\text{img}}$1

$\mathcal{E}_j^{\text{img}}$2

All leak verdicts are subjected to dual-annotator or ensemble LLM validation (Cohen’s $\mathcal{E}_j^{\text{img}}$3 for human agreement).

For explanation privacy, auditing is anchored by the Membership Leakage Score (MLS; TPR@FPR=0.1%) and Explanation-Utility Score ($\mathcal{E}_j^{\text{img}}$4, max change under $\mathcal{E}_j^{\text{img}}$5-bounded perturbations) (Hmida et al., 6 Jan 2026). Comparative results are produced across 15 techniques and 3 major datasets.

5. Best Practices and Deployment Guidelines

Semantically Precise Generation

• Apply DeLeaker by integrating dynamic, per-entity mask and attention reweighting in each transformer block during all relevant diffusion steps. Default FLUX .1-Dev settings: $\mathcal{E}_j^{\text{img}}$6, $\mathcal{E}_j^{\text{img}}$7, $\mathcal{E}_j^{\text{img}}$8, intervention over $\mathcal{E}_j^{\text{img}}$9 (Ventura et al., 16 Oct 2025).
• Regularly evaluate mitigation via human or advanced VLM-based benchmarks; monitor trade-offs in VQAScore, LPIPS, KID.

Memory Deletion Pipelines

• Instrument the IPG analyzer for each fact, categorize all memory nodes, and design cascade-del deletion APIs. Ensure addressable and linked nodes are transactionally deleted.
• Integrate content-aware semantic deletion as a post-deletion sweep, pruning images flagged by VLM similarity to forgotten facts.
• For highly robust compliance, deploy multi-VLM cross-validation and tune semantic-deletion thresholds to balance recall and false positives (Wang et al., 29 Jun 2026).

Explainability and Privacy

• Before production, audit explanation methods for MLS and tune parameters to minimal leakage (e.g., more integration steps in Integrated Gradients, adjusted superpixel count in LIME/SHAP).
• For non-parametric explainers, apply sequential clip→mask→noise transforms; select $e_i$0, $e_i$1, $e_i$2, and $e_i$3 based on the empirical 95th percentile of attributions.
• Always monitor $e_i$4; thresholds for utility loss are set at ≤3.3% across standard datasets (Hmida et al., 6 Jan 2026).

6. Open Challenges and Future Directions

Persistent leakage nodes (“Persistent” in the IPG) remain unsolved by current deletion APIs; these require research on privacy-aware vision encoders with disentangled representations or embedding scrubbing (projection out of $e_i$5-correlated directions in feature space). Cross-modal entanglement and text-correlation channels demand extended semantic auditing and more granular retrieval-time filtering. In explanation privacy, structural factors such as attribution sparsity and grad sensitivity dominate leakage risk rather than overfitting per se; parameter tuning and composite hardening sequences are most effective. Large-scale validation on in-the-wild datasets and further tuning of privacy–utility trade-offs are active needs for deployment at scale (Wang et al., 29 Jun 2026, Hmida et al., 6 Jan 2026).

7. Summary Table: DeLeaker System Properties

Application Domain	Core Mechanism	Key Metrics	Impact (vs. Baseline)
T2I Generation	Inference-time attention reweighting	Leakage mitigation %, LPIPS	55.8% mitigation, LPIPS 0.22
Multimodal Memory	IPG + semantic deletion	$e_i$6	12.0%→2.0%
Explanation Privacy	Clip→mask→noise hardening	MLS, $e_i$7	MLS up to 95%↓ ≤3.3% loss

The unifying principle across DeLeaker’s instantiations is systematic, traceable intervention at the representational, attentional, or attributional level to empirically minimize unwanted information transfer—backed by specialized, high-fidelity evaluation protocols and metrics (Ventura et al., 16 Oct 2025, Wang et al., 29 Jun 2026, Hmida et al., 6 Jan 2026).