DP-Fusion: Token-Level Differential Privacy

• DP-Fusion is a framework for token-level differential privacy that secures sensitive tokens in language model outputs through group-wise privacy guarantees.
• It employs a blended distributional inference technique, mixing public and private logits to maintain formal privacy bounds while optimizing output utility.
• Empirical evaluations reveal that DP-Fusion achieves a strong trade-off between privacy and language model performance, with competitive perplexity and reduced attack success rates.

DP-Fusion encompasses a family of algorithmic and architectural innovations in diverse fields, unified by the principle of fusion—whether of multimodal signals, computational dataflows, or information-theoretic guarantees. The term DP-Fusion has particular technical meanings in multimodal image fusion, high-throughput GPU architectures, and LLM privacy-preserving inference. This entry focuses on DP-Fusion in the sense of token-level Differentially Private Inference for LLMs, with context on adjacent usages where relevant.

1. Token-Level Differential Privacy and Core Guarantee

DP-Fusion, in the setting of LLMs, refers to a mechanism that ensures token-level (group-wise) differential privacy (DP) in generative LLM outputs, particularly when paraphrasing documents that may contain sensitive content such as personally identifiable information (PII). The foundational guarantee is as follows: For a document $D$ partitioned into a public token set $X_{\rm pub}$ and $m$ sensitive partitions $X_1, \ldots, X_m$, the mechanism $\mathsf{M}(D)$ generating a sequence of $T$ output tokens satisfies—per group $i$—the formal $(\varepsilon_i, \delta)$-DP definition:

$$\forall S,\ \forall D\stackrel{i}{\sim}D':\quad \Pr[\mathsf{M}(D)\in S] \le e^{\varepsilon_i}\Pr[\mathsf{M}(D')\in S]+\delta,$$

where $D\stackrel{i}{\sim}D'$ if $X_{\rm pub}$0 differs from $X_{\rm pub}$1 only by the tokens in group $X_{\rm pub}$2. The privacy loss $X_{\rm pub}$3 is analytically bounded, leveraging Rényi Differential Privacy (RDP) composition at order $X_{\rm pub}$4:

$X_{\rm pub}$5

This provides precise accounting of group influence over the generated output, allowing formal, interpretable privacy protection at the token level (Thareja et al., 6 Jul 2025).

2. Mechanism: Blended Distributional Inference

The DP-Fusion algorithm operates as follows:

1. Sensitive-Group Partitioning: Sensitive tokens are assigned to groups $X_{\rm pub}$6 (typically via NER or an oracle).
2. Parallel Inference: For each output token $X_{\rm pub}$7—given current context and query $X_{\rm pub}$8—compute standard (“public”) LLM logits $X_{\rm pub}$9 (context omitting $m$0) and, in parallel, “private-$m$1” logits $m$2 (context including $m$3) for all groups.
3. Mollified Blending: For each group $m$4, compute the maximal mixture coefficient $m$5 such that the symmetric Rényi divergence (order $m$6) between $m$7 and $m$8 is at most $m$9.
4. Averaging: The final next-token distribution is $X_1, \ldots, X_m$0.
5. Sampling: Draw the next token from $X_1, \ldots, X_m$1 and iterate for $X_1, \ldots, X_m$2 steps.

Algorithm 1 in (Thareja et al., 6 Jul 2025) formalizes this cycle, enabling fine-grained, groupwise privacy control.

3. Theoretical Properties and Conversion

The privacy guarantees derive from the following analytical principles:

• Monotonicity: The divergence $X_1, \ldots, X_m$3 is non-decreasing, ensuring that a unique maximal $X_1, \ldots, X_m$4 can be found (via bisection) that satisfies the divergence constraint.
• Group RDP and Composition: Each generation step incurs at most one group's RDP budget. Summing over $X_1, \ldots, X_m$5 steps, the total RDP per group is $X_1, \ldots, X_m$6. Standard RDP-to-$X_1, \ldots, X_m$7 conversion gives the stated DP bound.
• Explicit Utility/Privacy Trade-off: For $X_1, \ldots, X_m$8, all private tokens are perfectly hidden ($X_1, \ldots, X_m$9). As $\mathsf{M}(D)$0 increases, so do $\mathsf{M}(D)$1 and $\mathsf{M}(D)$2, trading privacy for improved utility.

This framework enables tractable, per-group privacy tracing over entire autoregressive inference trajectories, unlike baseline approaches which lack formal, compositional DP guarantees (Thareja et al., 6 Jul 2025).

4. Quantitative Trade-Offs: Privacy, Utility, and Overhead

Utility–privacy trade-off: Empirical results on the TAB-ECHR dataset with 100 annotated documents and 8 entity groups show that DP-Fusion, at tight privacy settings (e.g., $\mathsf{M}(D)$3, $\mathsf{M}(D)$4), achieves paraphrase perplexity (PPL) of $\mathsf{M}(D)$5, matching "public-only" baselines but with formal guarantees—while DP-Prompt and DP-Decoding baselines incur PPL $\mathsf{M}(D)$6. Slightly relaxed privacy settings ($\mathsf{M}(D)$7) recover PPL $\mathsf{M}(D)$8.

Privacy leakage: Attack success rate (ASR) on PII membership inference is sharply reduced by DP-Fusion (e.g., 26%) versus unprotected ($\mathsf{M}(D)$962%) and significantly outperforms DP-Decoding at equivalent utility.

Computational cost: Per token, $T$0 LLM forward passes are made (public + $T$1 private contexts). However, for $T$2, parallel batch execution enables execution slowdown as low as 1.5$T$3 relative to vanilla decoding on modern hardware.

5. Empirical Evaluation and Illustrative Scenarios

Comprehensive experiments validate both privacy and utility. Under strong privacy (low $T$4), DP-Fusion paraphrases strictly obfuscate all PII; as $T$5 increases, specific details can reappear within DP bounds. Judges (GPT-4) consistently preferred DP-Fusion outputs to DP-Prompt/Decoding at equivalent privacy, and preferred DP-Fusion to NER-only in nearly half the cases at moderate privacy.

Example paraphrasing illustrates this effect transparently: as the privacy parameter increases, successively more detail is permitted into the paraphrase, but formal privacy bounds are still enforced at each group level.

6. Methodological Distinctions, Limitations, and Adjacent Usages

Compared to alternatives, DP-Fusion uniquely provides (i) per-token, group-level DP; (ii) algorithmic mollification via blending, not hard masking or rejection sampling; and (iii) analytic composition for tractable privacy accounting.

Limitations include the linear runtime scaling with the number of privacy groups and memory needs proportional to $T$6. The approach assumes the availability of high-quality sensitive token groupings and sufficient hardware for parallel inference.

It is important to distinguish this DP-Fusion from uses of “DPF” and “DP-Fusion” in unrelated contexts—such as "Decomposition Pooling Fusion" for infrared-visible image fusion (Li et al., 2023), or "Data Path Fusion" in GPU database engines (Ozawa et al., 11 May 2026). In those domains, the fusion principle addresses signal decomposition or computational data paths, not privacy-preserved inference.

7. Broader Significance and Future Research

DP-Fusion establishes a rigorous framework for privatizing neural text generation with explicit, controllable trade-offs between semantic fidelity and measurable privacy leakage. The framework maps directly onto document summarization, anonymization, and any setting where LLM outputs must be provably insensitive to groups of sensitive input tokens. Future research may focus on optimized batch execution for large $T$7, adaptive groupings, or extension beyond text to multimodal outputs (Thareja et al., 6 Jul 2025).