Token-Level Dynamic Differential Privacy

• Token-Level Dynamic DP is a framework that quantifies and enforces privacy at the level of individual tokens, adapting noise based on sensitivity and temporal dependencies.
• It leverages dynamic allocation algorithms and advanced sensitivity analysis to calibrate per-token privacy budgets while managing cumulative leakage in sequential data.
• The approach is applied in language model inference and continual learning, achieving strong privacy guarantees without significantly compromising data utility.

Token-Level Dynamic Differential Privacy (DP) is a framework for quantifying and enforcing privacy guarantees at the finest granularity—individual tokens—during the dynamic release or processing of data. Unlike traditional approaches that provide a uniform privacy budget at the record or attribute level, token-level dynamic DP adapts the privacy protection for each token (e.g., word in a sequence), possibly at each time step, while rigorously accounting for temporal correlations, inter-token dependencies, and sensitivity heterogeneity. This approach leverages advanced algorithmic techniques, compositional theorems, and sensitivity analysis to calibrate noise or release strategies dynamically, addressing privacy risks in high-throughput, sequential, and continual scenarios.

1. Formalization and Motivation

In standard differential privacy, the output distribution of a randomized mechanism $M$ must be nearly indistinguishable between adjacent datasets differing in one record, yielding a guarantee: $$\Pr[M(d) \in S] \leq \exp(\epsilon) \Pr[M(d') \in S]$$ for all measurable $S$. In the token-level dynamic setting, the challenge is to extend this guarantee to each token in a sequence, potentially allowing $\epsilon_k$ to vary per token: $$\Pr[M_k(d) \in S] \leq \exp(\epsilon_k) \Pr[M_k(d') \in S]$$ with the global guarantee $\sum_k \epsilon_k \leq \epsilon_{\text{total}}$ (Danger, 2022). Furthermore, repeated or temporally correlated token outputs may compound privacy loss, necessitating refined composition and sensitivity analysis.

The driving motivation is to prevent privacy leakage from sensitive tokens in tasks such as LLM inference (Thareja et al., 6 Jul 2025), continual learning (Zhan et al., 16 Sep 2025), or sequential data release (Cao et al., 2017), while maintaining high data utility by reducing unnecessary noise added to less sensitive or non-target tokens.

2. Temporal Privacy Leakage and Correlated Sequences

Traditional DP mechanisms assume independent releases at each time point (or token), but practical data often exhibit strong temporal or sequential correlations. The concept of Temporal Privacy Leakage (TPL) (Cao et al., 2017) formalizes how privacy loss may accumulate or “leak” in the presence of such correlations.

TPL decomposes the privacy leakage into:

• Backward Privacy Leakage (BPL): Accumulated privacy loss from all previous releases.
• Forward Privacy Leakage (FPL): Influence of future releases on the current privacy loss.

These are precisely characterized by: $$\mathrm{TPL}(\mathcal{M}^t) = \mathrm{BPL}(\mathcal{M}^t) + \mathrm{FPL}(\mathcal{M}^t) - \mathrm{PL}_0(\mathcal{M}^t)$$ where $\mathrm{PL}_0$ is the event-level privacy leakage. When token-level outputs are temporally correlated—modeled via time-homogeneous Markov chains with transition matrices $P_i^B$ (backward) and $P_i^F$ (forward)—TPL must be computed recursively, with leakage potentially increasing over time, but under practical conditions, it admits a supremum [(Cao et al., 2017), Theorem sup].

Applying these principles to token streams, each token's privacy cost depends not only on its immediate release but also on its dependencies with previous and future tokens. Naive per-token DP does not adequately bound cumulative leakage in correlated data; dynamic, correlation-aware calibration is required.

3. Algorithmic Techniques for Dynamic Token-Level DP

Dynamic token-level DP leverages several algorithmic innovations to compute and enforce privacy guarantees efficiently, even under complex dependencies:

Dynamic Allocation Algorithms

• Linear-Fractional Programming (LFP): Temporal privacy loss (BPL, FPL) reduces to LFP problems that maximize ratios under bounded ratios between vector elements (Cao et al., 2017). Efficient algorithms (polynomial-time, quasi-quadratic, and sublinear via precomputed piecewise functions and binary search) enable TPL computation even for large state spaces.

Dynamic Budget Calibration

• Constant Upper-Bound Calibration: Assigns a privacy budget $\epsilon_t$ at each time step such that the supremum of TPL remains under a target $\alpha$.
• Quantified Dynamic Allocation: Optimally distributes the privacy budget over time via closed-form equations, possibly allocating higher budgets at sequence endpoints and lower in the interior for minimal utility loss.

Adaptive Tokenwise Sensitivity and Noise

• Score-Based Sensitivity: Sensitivities for each token $t_i$ are computed as a function of uncertainty (e.g., $-\log P_\theta(t_i|\text{context})$) and task-specific discriminativeness, yielding a fused score $s_i$ (Zhan et al., 16 Sep 2025).
• Dynamic Noise Injection: Privacy budgets $\epsilon_i$ for each token are allocated inversely to sensitivity, and noise is injected at the embedding or output distribution level, following $\mathcal{N}(0, \sigma_i^2 I)$ with $\sigma_i = C\sqrt{2\log(1.25/\delta)}/\epsilon_i$.

4. Practical Mechanisms and System Architectures

The deployment of token-level dynamic DP in real systems encompasses robust mechanism design, context-sensitive calibration, ongoing privacy accounting, and practical implementation considerations:

Token-Level Differentially Private Inference

• DP-Fusion: Partitions input context into public and multiple sensitive token groups, runs the LLM on each group, and blends output distributions using a mixing parameter $\lambda$ constrained by a Rényi divergence threshold (Thareja et al., 6 Jul 2025). The resulting blended output provides tight, per-token or per-group $(\epsilon, \delta)$-DP guarantees while maintaining utility (e.g., low perplexity in document privatization).

Continual Learning with Memory Sculpting

• PeCL Framework: Integrates token-level DP mechanisms with privacy-guided memory sculpting, wherein parameter regularization and targeted unlearning (gradient reversal applied to high-sensitivity tokens) prevents sensitive content from cumulative memorization across tasks (Zhan et al., 16 Sep 2025). Regularization strength is modulated by the average token sensitivity in each task.

Hierarchical and Streaming Methods

• Stream Data Release: Methods such as hierarchical noise addition with online consistency and smoothing are applied to continuous streams. Truncation thresholds are optimized using exponential mechanisms to minimize the bias-variance tradeoff, and online algorithms propagate consistent, DP-protected stream outputs (Wang et al., 2020).

Attribute/Token-Granular Guarantees

• Partial/Per-token DP: Mechanisms are designed to ensure that if $k$ tokens (attributes) differ between two sequences, the output distributions are within $\exp(\epsilon_0 \cdot k)$, enabling strong guarantees even as overall sequence length grows (Ghazi et al., 2022).

5. Sensitivity Analysis and Targeted Protection

Token-level DP enables fine-grained sensitivity analysis, yielding more nuanced privacy-utility tradeoffs:

• Partial Sensitivity Analysis: For a function $f:\mathbb{R}^m \to \mathbb{R}$, the partial sensitivity of each input feature $x_j$ is given as $(\partial f/\partial x_j) / \|\nabla f(x)\|_2$ (Mueller et al., 2021). Symbolic automatic differentiation provides analytic, interpretable assessments of each token's (or feature's) privacy impact.
• Weighted Noise Injection: Mechanisms such as weighted randomized response (as in TLDP for tensors (Yuan et al., 25 Feb 2025)) adjust the retention probability or noise scale for each component/token according to application-defined or data-driven sensitivity weights, enabling selective strengthening of privacy.

6. Applications and Empirical Performance

Token-level dynamic DP has been validated in scenarios requiring both privacy and high utility:

Application Domain	Mechanism/Method	Key Results/Findings
Document Privatization (LLMs)	DP-Fusion (Thareja et al., 6 Jul 2025)	Low perplexity paraphrasing; strict PII privacy
Continual Seq. Learning	PeCL (Zhan et al., 16 Sep 2025)	High accuracy and low forgetting; robust privacy
Tensor Data Privatization	TLDP (Yuan et al., 25 Feb 2025)	Order-of-magnitude better F1 compared to baselines
Streaming Data Release	Hierarchical, Dynamic Methods	<em\>6–10x</em> lower error than previous LDP methods
Token-wise Mechanisms	Partial DP (Ghazi et al., 2022), TPL (Cao et al., 2017)	Fine-grained, additive privacy bounds

In LLMs, empirically, token-level DP enables strong control over PII leakage with minimal degradation in fluency or content preservation; in continual learning, token-wise dynamic noise preserves both historic knowledge and privacy.

7. Implementation Challenges and Limitations

Several challenges persist:

• Computation and Scalability: Methods such as DP-Fusion (Thareja et al., 6 Jul 2025) scale the number of model runs with the count of privacy groups; memory and compute can be limiting.
• Correlation Modeling: Markov models are a practical assumption for temporal dependence, but real-world token dependencies may exhibit higher order or non-Markovian effects (Cao et al., 2017).
• Parameter Selection: Mapping between Rényi divergence budgets, $\epsilon$, and perceived privacy is nontrivial, and often context dependent.
• Oracle Reliance: NER or privacy-oracle accuracy in group assignment is critical; misclassification can result in unanticipated leakage.
• Redundancy and Dependency: In highly correlated (e.g., natural language) data, naive composition may underestimate privacy risk; adaptive or redundancy-aware mechanisms may be necessary.

Possible remedies involve accelerated algorithms (precomputing piecewise privacy loss functions), adaptive or data-driven sensitivity measures, and ongoing theoretical advancements in privacy accounting under complex dependencies.

---

Token-level dynamic differential privacy constitutes a comprehensive framework for privacy protection in temporally correlated, high-throughput, and semantically heterogeneous data streams. By incorporating dynamic, per-token calibration—supported by advanced sensitivity analysis, efficient privacy loss computation (e.g., via linear-fractional programming), and application-specific mechanisms—this approach achieves strong, granular privacy guarantees with empirically validated preservation of utility across diverse real-world applications.