Add-Remove Heterogeneous Differential Privacy (AHDP)

• AHDP is a set of differential privacy formulations that replace a global budget with structured, context-aware guarantees via add/remove neighboring datasets.
• It encompasses diverse models including graph-based, coordinate-wise, and correlation-aware settings, each employing unique optimization techniques and trade-offs.
• AHDP frameworks enable practical implementations with efficient algorithms, offering improved utility and tailored privacy protection under varying demands.

Add-remove Heterogeneous Differential Privacy (AHDP) denotes a family of differential privacy formulations that retain add/remove-style neighboring datasets while allowing privacy protection to vary across users, dataset pairs, or typed records. In the cited literature, the term appears in closely related but not identical formal settings: graph-based heterogeneous DP with edge-dependent privacy parameters and utility-optimal extension from a boundary set (Torkamani et al., 2022); coordinate-wise heterogeneous privacy demands for mean estimation under add/remove adjacency (Chaudhuri et al., 2023); and a correlation-aware framework in which each record is a pair $(x,\epsilon)$ and privacy is measured by a weighted add–remove distance that jointly accounts for data and privacy demand (Chaudhuri et al., 2 Sep 2025). Across these formulations, the common objective is to replace a single global privacy budget with structured heterogeneity while preserving operational DP guarantees.

1. Formal scope and principal formulations

The literature uses AHDP to describe heterogeneous privacy under add/remove semantics, but the primitive object that carries heterogeneity differs by paper. In one line of work, datasets are vertices of a graph and heterogeneity is attached to edges through a symmetric function $\varepsilon:E\to[0,\infty)$ (Torkamani et al., 2022). In another, datasets are vectors $\mathbf X\in\mathcal X^n$ and user $i$ carries an individual budget $\epsilon_i$ that governs all neighboring dataset pairs differing only in coordinate $i$ (Chaudhuri et al., 2023). In the correlation-aware formulation, each input is a pair $$(x,\epsilon)\in\mathcal W\subseteq \mathcal X\times \mathbb R_{\ge 0}$$, datasets are multisets of such pairs, and privacy is defined through a weighted add–remove metric $d_\alpha$ (Chaudhuri et al., 2 Sep 2025).

Formulation	Dataset model	Heterogeneity object
Graph-based heterogeneous DP	Graph $\mathcal G=(V,E)$ with datasets as vertices	Edge budget $\varepsilon(u,v)$
Coordinate-wise heterogeneous DP	$\varepsilon:E\to[0,\infty)$0	User budget $\varepsilon:E\to[0,\infty)$1
Correlation-aware AHDP	Multiset of $\varepsilon:E\to[0,\infty)$2 pairs	Weight $\varepsilon:E\to[0,\infty)$3

For the graph-based definition, a randomized mechanism $\varepsilon:E\to[0,\infty)$4 is $\varepsilon:E\to[0,\infty)$5-DP if for every edge $\varepsilon:E\to[0,\infty)$6 and every event $\varepsilon:E\to[0,\infty)$7,

$\varepsilon:E\to[0,\infty)$8

For the coordinate-wise definition, a mechanism $\varepsilon:E\to[0,\infty)$9 satisfies $\mathbf X\in\mathcal X^n$0-Heterogeneous DP if, for every $\mathbf X\in\mathcal X^n$1, every pair of datasets differing only in coordinate $\mathbf X\in\mathcal X^n$2, and every event $\mathbf X\in\mathcal X^n$3,

$\mathbf X\in\mathcal X^n$4

For the correlation-aware definition, letting $\mathbf X\in\mathcal X^n$5 denote the count of type $\mathbf X\in\mathcal X^n$6 in dataset $\mathbf X\in\mathcal X^n$7 and fixing $\mathbf X\in\mathcal X^n$8, the add–remove distance is

$\mathbf X\in\mathcal X^n$9

and $i$0 is $i$1-AHDP if for every $i$2 and measurable $i$3,

$i$4

These definitions share add/remove structure but not a single canonical state space. This suggests that AHDP is best understood as a heterogeneous-privacy design pattern rather than a unique formalism.

2. Graph-based AHDP and utility-optimal boundary extension

In "Heterogeneous Differential Privacy via Graphs" (Torkamani et al., 2022), the starting point is a simple undirected graph $i$5 in which each vertex is a database and $i$6 means that $i$7 can be obtained from $i$8 by adding or removing exactly one record. For a binary true query $i$9, the boundary set $\epsilon_i$0 consists of those $\epsilon_i$1 for which there exists a neighbor $\epsilon_i$2 with $\epsilon_i$3. The mechanism is first specified on this boundary, with Bernoulli output probabilities

$\epsilon_i$4

and the central problem is to extend $\epsilon_i$5 from $\epsilon_i$6 to all of $\epsilon_i$7 while preserving heterogeneous DP and maximizing pointwise utility.

The key device is the strongest induced DP condition. Although DP is imposed only on edges, path chaining yields induced upper bounds on $\epsilon_i$8 from a starting value $\epsilon_i$9. Along an edge $i$0, the paper records the two inequalities

$i$1

and

$i$2

Composing these linear maps along a path $i$3 gives an induced upper bound $i$4, and the strongest induced bound is

$i$5

Compatibility of boundary probabilities is then checked by requiring

$i$6

for all $i$7.

If the boundary assignment is compatible, the optimal extension to any $i$8 is

$i$9

and

$$(x,\epsilon)\in\mathcal W\subseteq \mathcal X\times \mathbb R_{\ge 0}$$0

The paper states that this extension is the unique maximizer of $$(x,\epsilon)\in\mathcal W\subseteq \mathcal X\times \mathbb R_{\ge 0}$$1 at each $$(x,\epsilon)\in\mathcal W\subseteq \mathcal X\times \mathbb R_{\ge 0}$$2 subject to all heterogeneous DP constraints. It also gives a path-graph closed form: on a simple path $$(x,\epsilon)\in\mathcal W\subseteq \mathcal X\times \mathbb R_{\ge 0}$$3 with budgets $$(x,\epsilon)\in\mathcal W\subseteq \mathcal X\times \mathbb R_{\ge 0}$$4 and $$(x,\epsilon)\in\mathcal W\subseteq \mathcal X\times \mathbb R_{\ge 0}$$5, the unique optimal extension has a two-regime formula determined by a threshold index $$(x,\epsilon)\in\mathcal W\subseteq \mathcal X\times \mathbb R_{\ge 0}$$6, and the resulting pathwise bias obeys

$$(x,\epsilon)\in\mathcal W\subseteq \mathcal X\times \mathbb R_{\ge 0}$$7

Computationally, each $$(x,\epsilon)\in\mathcal W\subseteq \mathcal X\times \mathbb R_{\ge 0}$$8 can be computed in polynomial time by a Dijkstra-style shortest-path routine in $$(x,\epsilon)\in\mathcal W\subseteq \mathcal X\times \mathbb R_{\ge 0}$$9. Testing compatibility on the boundary requires $d_\alpha$0, extending to all vertices requires $d_\alpha$1, and the worst-case complexity is $d_\alpha$2. The authors note that the algorithm relies only on shortest-path subroutines and is directly implementable in any graph library; when $d_\alpha$3 is much smaller than $d_\alpha$4, the effective runtime is correspondingly lower (Torkamani et al., 2022).

3. Coordinate-wise heterogeneous privacy demands and AHDP mean estimation

"Mean Estimation Under Heterogeneous Privacy Demands" studies bounded univariate mean estimation under user-specific privacy levels (Chaudhuri et al., 2023). The data domain is $d_\alpha$5, datasets are vectors $d_\alpha$6, and heterogeneous privacy is indexed by $d_\alpha$7, typically ordered so that $d_\alpha$8. The privacy-loss formulation is coordinate specific: $d_\alpha$9

The paper’s concrete mechanism is the Affine Differentially Private Mean (ADPM). After sorting the budgets, it constructs a sequence $\mathcal G=(V,E)$0 recursively: $\mathcal G=(V,E)$1 where $\mathcal G=(V,E)$2 and $\mathcal G=(V,E)$3 are updated cumulatively as $\mathcal G=(V,E)$4 and $\mathcal G=(V,E)$5. If

$\mathcal G=(V,E)$6

the mechanism returns the constant estimator $\mathcal G=(V,E)$7. Otherwise it sets

$\mathcal G=(V,E)$8

draws $\mathcal G=(V,E)$9, and outputs

$\varepsilon(u,v)$0

The complexity is $\varepsilon(u,v)$1 time and $\varepsilon(u,v)$2 memory, with near-linear time possible when the $\varepsilon(u,v)$3 come from a small discrete set.

The paper ties privacy calibration directly to coordinate sensitivity. For the affine estimator $\varepsilon(u,v)$4, the change induced by varying $\varepsilon(u,v)$5 over $\varepsilon(u,v)$6 is $\varepsilon(u,v)$7. A Laplace mechanism with scale $\varepsilon(u,v)$8 is $\varepsilon(u,v)$9-DP at coordinate $\varepsilon:E\to[0,\infty)$00 if $\varepsilon:E\to[0,\infty)$01, so choosing

$\varepsilon:E\to[0,\infty)$02

suffices. In ADPM, the construction ensures $\varepsilon:E\to[0,\infty)$03, hence $\varepsilon:E\to[0,\infty)$04 enforces all coordinate-wise privacy constraints simultaneously. The summary emphasizes that no “standard” composition theorem is needed because a single global noise scale is calibrated against all user budgets.

The utility analysis is minimax. Defining

$\varepsilon:E\to[0,\infty)$05

the minimax mean-squared error satisfies

$\varepsilon:E\to[0,\infty)$06

for universal constants $\varepsilon:E\to[0,\infty)$07, and ADPM achieves the upper bound with $\varepsilon:E\to[0,\infty)$08 up to small constant factors. The optimization underlying the upper bound is

$\varepsilon:E\to[0,\infty)$09

with a water-filling argument, quasi-convexity, and KKT conditions yielding the recursion implemented by ADPM.

A central structural phenomenon is saturation. If $\varepsilon:E\to[0,\infty)$10 is the smallest index such that

$\varepsilon:E\to[0,\infty)$11

then for $\varepsilon:E\to[0,\infty)$12 one has $\varepsilon:E\to[0,\infty)$13, while for $\varepsilon:E\to[0,\infty)$14 the recursion saturates and all $\varepsilon:E\to[0,\infty)$15 become equal to the same common value independent of $\varepsilon:E\to[0,\infty)$16. The paper interprets this as a “privacy-for-free” effect: once the most stringent users determine the error rate, increasing the budgets of less stringent users does not further improve the minimax rate (Chaudhuri et al., 2023).

4. Correlation-aware AHDP and its operational meaning

"Managing Correlations in Data and Privacy Demand" argues that conventional HDP can fail when user data and privacy demand are correlated (Chaudhuri et al., 2 Sep 2025). In that setting, holding $\varepsilon:E\to[0,\infty)$17 fixed while varying only $\varepsilon:E\to[0,\infty)$18 is insufficient because the pair $\varepsilon:E\to[0,\infty)$19 may itself be private and correlated. The proposed remedy is a correlation-aware AHDP framework in which each tuple $\varepsilon:E\to[0,\infty)$20 is treated as an atomic type and neighboring changes are measured on multisets of such pairs.

Formally, with dataset $\varepsilon:E\to[0,\infty)$21 and count function $\varepsilon:E\to[0,\infty)$22, the weight function $\varepsilon:E\to[0,\infty)$23 is required later to satisfy $\varepsilon:E\to[0,\infty)$24 for all $\varepsilon:E\to[0,\infty)$25. The add–remove distance is

$\varepsilon:E\to[0,\infty)$26

A randomized mechanism is $\varepsilon:E\to[0,\infty)$27-AHDP if

$\varepsilon:E\to[0,\infty)$28

for every $\varepsilon:E\to[0,\infty)$29 and measurable $\varepsilon:E\to[0,\infty)$30. If $\varepsilon:E\to[0,\infty)$31 is homogeneous and $\varepsilon:E\to[0,\infty)$32, this recovers add–remove $\varepsilon:E\to[0,\infty)$33-DP. The summary also states that post-processing and composition hold exactly as in ordinary DP, with $\varepsilon:E\to[0,\infty)$34.

The paper supplies an operational interpretation through binary hypothesis testing. Given observations $\varepsilon:E\to[0,\infty)$35 and hypotheses

$\varepsilon:E\to[0,\infty)$36

for a rejection region $\varepsilon:E\to[0,\infty)$37 define

$\varepsilon:E\to[0,\infty)$38

Then $\varepsilon:E\to[0,\infty)$39 is $\varepsilon:E\to[0,\infty)$40-AHDP if and only if

$\varepsilon:E\to[0,\infty)$41

The summary further states that, in AHDP, if an adversary might be distinguishing $\varepsilon:E\to[0,\infty)$42 from $\varepsilon:E\to[0,\infty)$43, her power is at most $\varepsilon:E\to[0,\infty)$44, and that for a sequence of threat models with $\varepsilon:E\to[0,\infty)$45 unseen points one obtains

$\varepsilon:E\to[0,\infty)$46

where $\varepsilon:E\to[0,\infty)$47.

A major contrast with standard HDP is explicit: the summary states that standard HDP can completely fail under correlations, and that one can leak the entire dataset in some contrived but valid HDP mechanism. The AHDP reformulation is presented as robust to such correlations because the privacy guarantee is indexed by the combined data–privacy type rather than data alone (Chaudhuri et al., 2 Sep 2025).

5. Universal mechanisms under correlation-aware AHDP

The correlation-aware paper develops universal AHDP mechanisms, meaning mechanisms that are $\varepsilon:E\to[0,\infty)$48-AHDP for every finite $\varepsilon:E\to[0,\infty)$49 as soon as the chosen weights satisfy $\varepsilon:E\to[0,\infty)$50 (Chaudhuri et al., 2 Sep 2025). The summary gives four concrete families.

The first is a Sampling Mechanism, attributed to Jørgensen et al. ’15. Given dataset $\varepsilon:E\to[0,\infty)$51, thresholds $\varepsilon:E\to[0,\infty)$52, and $\varepsilon:E\to[0,\infty)$53, each $\varepsilon:E\to[0,\infty)$54 is included in a subsample $\varepsilon:E\to[0,\infty)$55 independently with probability $\varepsilon:E\to[0,\infty)$56, after which any homogeneous $\varepsilon:E\to[0,\infty)$57-DP algorithm is run on $\varepsilon:E\to[0,\infty)$58. The resulting mechanism is stated to be $\varepsilon:E\to[0,\infty)$59-AHDP.

The second family covers linear-counting queries such as sums, counts, and histograms. For $\varepsilon:E\to[0,\infty)$60 with $\varepsilon:E\to[0,\infty)$61 and

$\varepsilon:E\to[0,\infty)$62

the mechanism is

$\varepsilon:E\to[0,\infty)$63

Its add–remove sensitivity in the weighted metric is exactly $\varepsilon:E\to[0,\infty)$64, which justifies the Laplace scale, and any choice $\varepsilon:E\to[0,\infty)$65 yields universal AHDP.

The third family is mean estimation via two queries for $\varepsilon:E\to[0,\infty)$66. With two weight functions $\varepsilon:E\to[0,\infty)$67, the numerator and denominator are

$\varepsilon:E\to[0,\infty)$68

and

$\varepsilon:E\to[0,\infty)$69

and the output is

$\varepsilon:E\to[0,\infty)$70

By composition, $\varepsilon:E\to[0,\infty)$71 is $\varepsilon:E\to[0,\infty)$72-AHDP and $\varepsilon:E\to[0,\infty)$73 is $\varepsilon:E\to[0,\infty)$74-AHDP, so $\varepsilon:E\to[0,\infty)$75 is $\varepsilon:E\to[0,\infty)$76-AHDP.

The fourth family is linear regression. For data

$\varepsilon:E\to[0,\infty)$77

choose weights $\varepsilon:E\to[0,\infty)$78, form $\varepsilon:E\to[0,\infty)$79, $\varepsilon:E\to[0,\infty)$80, and $\varepsilon:E\to[0,\infty)$81, then compute

$\varepsilon:E\to[0,\infty)$82

and output $\varepsilon:E\to[0,\infty)$83. The summary states that each entry of $\varepsilon:E\to[0,\infty)$84 and $\varepsilon:E\to[0,\infty)$85 has add–remove sensitivity $\varepsilon:E\to[0,\infty)$86 in $\varepsilon:E\to[0,\infty)$87, so the Laplace scale yields $\varepsilon:E\to[0,\infty)$88-AHDP.

The empirical section evaluates these mechanisms on two GPT-4o-generated synthetic datasets and one real dataset. For mean estimation, the synthetic data contain approximately $\varepsilon:E\to[0,\infty)$89 samples of $\varepsilon:E\to[0,\infty)$90 with correlation approximately $\varepsilon:E\to[0,\infty)$91; Sampling Mechanism $\varepsilon:E\to[0,\infty)$92 yields $\varepsilon:E\to[0,\infty)$93 as $\varepsilon:E\to[0,\infty)$94, whereas fixed-weight methods incur a constant bias. For frequency estimation, the data contain $\varepsilon:E\to[0,\infty)$95 GPT-generated tuples across $\varepsilon:E\to[0,\infty)$96 education levels and $\varepsilon:E\to[0,\infty)$97 $\varepsilon:E\to[0,\infty)$98-levels, with a $\varepsilon:E\to[0,\infty)$99 test rejecting independence; Sampling $\mathbf X\in\mathcal X^n$00 outperforms but does not vanish as $\mathbf X\in\mathcal X^n$01, while linear-query methods have nontrivial error. For linear regression on California Housing, with $\mathbf X\in\mathcal X^n$02 training and $\mathbf X\in\mathcal X^n$03 test samples and $\mathbf X\in\mathcal X^n$04, weighted least squares with $\mathbf X\in\mathcal X^n$05 has the lowest error among universal AHDP schemes and approaches the nonprivate baseline as $\mathbf X\in\mathcal X^n$06 grows. The same summary reports computational costs of $\mathbf X\in\mathcal X^n$07 for sums and histograms and $\mathbf X\in\mathcal X^n$08 for regression.

6. Conceptual issues, relationships, and open directions

Several technical issues recur across the AHDP literature. One is the relation to personalized DP. The graph-based paper states that allowing $\mathbf X\in\mathcal X^n$09 to be a function of neighboring datasets recovers an earlier definition of personalized DP as a special case (Torkamani et al., 2022). Another is the status of correlation between data and privacy demand. The correlation-aware paper makes this a central objection to standard HDP, asserting that frameworks which vary data while holding privacy labels fixed can be inadequate when $\mathbf X\in\mathcal X^n$10 is correlated and private (Chaudhuri et al., 2 Sep 2025).

A common misconception is that heterogeneity merely means replacing a single $\mathbf X\in\mathcal X^n$11 with a vector $\mathbf X\in\mathcal X^n$12 and then applying standard intuitions unchanged. The papers collectively indicate that this is incomplete. In the graph formulation, heterogeneity propagates through paths and induced constraints, so compatibility and optimal extension are global graph problems rather than local budget substitutions. In mean estimation, the minimax rate is controlled by

$\mathbf X\in\mathcal X^n$13

which encodes a prefix effect rather than simple averaging of budgets. In the correlation-aware formulation, the relevant protected object is the joint type $\mathbf X\in\mathcal X^n$14, not merely the datum $\mathbf X\in\mathcal X^n$15.

The papers also identify distinct bias–variance trade-offs. In graph-based AHDP, utility is framed as maximizing $\mathbf X\in\mathcal X^n$16 at each vertex subject to heterogeneous DP, and the extension is pointwise optimal. In the mean-estimation setting, ADPM balances $\mathbf X\in\mathcal X^n$17 against $\mathbf X\in\mathcal X^n$18, yielding the saturation phenomenon in which larger-budget users may receive more privacy than requested without utility loss (Chaudhuri et al., 2023). In universal AHDP mechanisms for correlated data, linear-query methods trade bias from down-weighting private points against DP noise, while sampling trades subsampling bias against DP noise; the summary explicitly notes that the choice of $\mathbf X\in\mathcal X^n$19 or $\mathbf X\in\mathcal X^n$20 must be tuned (Chaudhuri et al., 2 Sep 2025).

Open problems are stated in task-specific terms. For heterogeneous mean estimation these include tighter heterogeneous-DP composition theorems, data-dependent tuning of weights when $\mathbf X\in\mathcal X^n$21 are private, extensions to sub-Gaussian or heavy-tailed domains, multivariate AHDP, and practical implementations in streaming or federated settings (Chaudhuri et al., 2023). For correlation-aware AHDP, the listed directions include $\mathbf X\in\mathcal X^n$22-AHDP or Rényi-AHDP variants via Gaussian mechanisms, task-specific mechanisms beyond linear models, membership-inference stress tests, characterization of minimal bias under realistic distributional assumptions, and optimal budget splitting between numerator and denominator in mean estimation (Chaudhuri et al., 2 Sep 2025).

Taken together, these works present AHDP as a technically heterogeneous research area organized around a shared principle: add/remove privacy guarantees can be individualized, but the correct formal object for that individualization depends on whether the heterogeneity is attached to graph edges, user coordinates, or joint data–privacy types. This suggests that the central unresolved question is not whether heterogeneous privacy is possible, but which add/remove formalism is appropriate for the statistical, structural, and adversarial assumptions of a given problem.