--- title: Add-Remove Heterogeneous Differential Privacy (AHDP) url: https://www.emergentmind.com/topics/add-remove-heterogeneous-differential-privacy-ahdp type: topic --- # Add-Remove Heterogeneous Differential Privacy (AHDP) Add-remove Heterogeneous Differential Privacy (AHDP) denotes a family of differential privacy formulations that retain add/remove-style neighboring datasets while allowing privacy protection to vary across users, dataset pairs, or typed records. In the cited literature, the term appears in closely related but not identical formal settings: graph-based heterogeneous DP with edge-dependent privacy parameters and utility-optimal extension from a boundary set [2203.15429]; coordinate-wise heterogeneous privacy demands for mean estimation under add/remove adjacency [2310.13137]; and a correlation-aware framework in which each record is a pair $(x,\epsilon)$ and privacy is measured by a weighted add–remove distance that jointly accounts for data and privacy demand [2509.02856]. Across these formulations, the common objective is to replace a single global privacy budget with structured heterogeneity while preserving operational DP guarantees. ## 1. Formal scope and principal formulations The literature uses AHDP to describe heterogeneous privacy under add/remove semantics, but the primitive object that carries heterogeneity differs by paper. In one line of work, datasets are vertices of a graph and heterogeneity is attached to edges through a symmetric function $\varepsilon:E\to[0,\infty)$ [2203.15429]. In another, datasets are vectors $\mathbf X\in\mathcal X^n$ and user $i$ carries an individual budget $\epsilon_i$ that governs all neighboring dataset pairs differing only in coordinate $i$ [2310.13137]. In the correlation-aware formulation, each input is a pair $(x,\epsilon)\in\mathcal W\subseteq \mathcal X\times \mathbb R_{\ge 0}$, datasets are multisets of such pairs, and privacy is defined through a weighted add–remove metric $d_\alpha$ [2509.02856]. | Formulation | Dataset model | Heterogeneity object | |---|---|---| | Graph-based heterogeneous DP | Graph $\mathcal G=(V,E)$ with datasets as vertices | Edge budget $\varepsilon(u,v)$ | | Coordinate-wise heterogeneous DP | $\mathbf X\in\mathcal X^n$ | User budget $\epsilon_i$ | | Correlation-aware AHDP | Multiset of $(x,\epsilon)$ pairs | Weight $\alpha(x,\epsilon)\le \epsilon$ | For the graph-based definition, a randomized mechanism $\mathcal M:V\to Q$ is $\varepsilon(\cdot)$-DP if for every edge $(u,v)\in E$ and every event $S\subseteq Q$, \[ \Pr[\mathcal M(u)\in S]\le \exp(\varepsilon(u,v))\Pr[\mathcal M(v)\in S]. \] For the coordinate-wise definition, a mechanism $M:\mathcal X^n\to\mathcal Y$ satisfies $\boldsymbol\epsilon$-Heterogeneous DP if, for every $i\in[n]$, every pair of datasets differing only in coordinate $i$, and every event $S\subset \mathcal Y$, \[ \Pr[M(\mathbf X)\in S]\le \exp(\epsilon_i)\Pr[M(\mathbf X')\in S]. \] For the correlation-aware definition, letting $h_D(w)$ denote the count of type $w$ in dataset $D$ and fixing $\alpha:\mathcal W\to\mathbb R_{\ge 0}$, the add–remove distance is \[ d_\alpha(D,D')=\sum_{w\in\mathcal W}\alpha(w)\,|h_D(w)-h_{D'}(w)|, \] and $M:\mathcal S(\mathcal W)\to\mathcal Y$ is $\alpha$-AHDP if for every $D,D'$ and measurable $S\subseteq\mathcal Y$, \[ \left|\log \frac{\Pr[M(D)\in S]}{\Pr[M(D')\in S]}\right|\le d_\alpha(D,D'). \] These definitions share add/remove structure but not a single canonical state space. This suggests that AHDP is best understood as a heterogeneous-privacy design pattern rather than a unique formalism. ## 2. Graph-based AHDP and utility-optimal boundary extension In "Heterogeneous Differential Privacy via Graphs" [2203.15429], the starting point is a simple undirected graph $\mathcal G=(V,E)$ in which each vertex is a database and $(u,v)\in E$ means that $v$ can be obtained from $u$ by adding or removing exactly one record. For a binary true query $T:V\to\{1,2\}$, the boundary set $\partial_T(\mathcal G)$ consists of those $u\in V$ for which there exists a neighbor $v$ with $T(v)\ne T(u)$. The mechanism is first specified on this boundary, with Bernoulli output probabilities \[ \Pr[\mathcal M(u)=T(u)]=p(u),\qquad \Pr[\mathcal M(u)\ne T(u)]=1-p(u), \] and the central problem is to extend $p(\cdot)$ from $\partial_T(\mathcal G)$ to all of $V$ while preserving heterogeneous DP and maximizing pointwise utility. The key device is the strongest induced DP condition. Although DP is imposed only on edges, path chaining yields induced upper bounds on $p(v)$ from a starting value $p(u)=\alpha$. Along an edge $(x,y)$, the paper records the two inequalities \[ p(y)\le \exp(\varepsilon(x,y))\,p(x), \] and \[ p(y)\le \frac{p(x)-1+\exp(\varepsilon(x,y))}{\exp(\varepsilon(x,y))}. \] Composing these linear maps along a path $\rho$ gives an induced upper bound $U_{\rho,\alpha}(v)$, and the strongest induced bound is \[ SIB_{u\to v}(\alpha)\coloneqq \min_{\rho\in\mathcal P(u,v)}U_{\rho,\alpha}(v). \] Compatibility of boundary probabilities is then checked by requiring \[ p(v)\le SIB_{u\to v}(p(u)) \] for all $u,v\in \partial_T(\mathcal G)$. If the boundary assignment is compatible, the optimal extension to any $w\in V\setminus \partial_T(\mathcal G)$ is \[ p(w)=\min_{u\in\partial_T(\mathcal G)} SIB_{u\to w}(p(u)),\quad \text{if }T(w)=1, \] and \[ p(w)=1-\min_{u\in\partial_T(\mathcal G)} SIB_{u\to w}(1-p(u)),\quad \text{if }T(w)=2. \] The paper states that this extension is the unique maximizer of $\Pr[\mathcal M(v)=T(v)]$ at each $v$ subject to all heterogeneous DP constraints. It also gives a path-graph closed form: on a simple path $v_0\to v_1\to\cdots\to v_n$ with budgets $\varepsilon_0,\ldots,\varepsilon_{n-1}$ and $p(v_0)=\alpha$, the unique optimal extension has a two-regime formula determined by a threshold index $\tau$, and the resulting pathwise bias obeys \[ \mathrm{bias}(v_i)=\left|p^*(v_i)-1_{T(v_i)=1}\right| \le \max\left\{e^{-\sum_{j\frac14, \] the mechanism returns the constant estimator $0$. Otherwise it sets \[ w_i=\frac{r_i}{L_1},\qquad \eta=\frac{1}{L_1}, \] draws $N\sim \mathrm{Laplace}(\mathrm{scale}=\eta)$, and outputs \[ M(\mathbf X)=\sum_{i=1}^n w_i x_i + N. \] The complexity is $O(n\log n)$ time and $O(n)$ memory, with near-linear time possible when the $\epsilon_i$ come from a small discrete set. The paper ties privacy calibration directly to coordinate sensitivity. For the affine estimator $\sum_i w_i x_i$, the change induced by varying $x_i$ over $[-1/2,1/2]$ is $\Delta_i=|w_i|$. A Laplace mechanism with scale $\eta$ is $\epsilon_i$-DP at coordinate $i$ if $w_i/\eta\le \epsilon_i$, so choosing \[ \eta=\max_i \{w_i/\epsilon_i\} \] suffices. In ADPM, the construction ensures $r_i\le \epsilon_i\cdot \sum_j r_j$, hence $\eta=1/\sum_j r_j$ enforces all coordinate-wise privacy constraints simultaneously. The summary emphasizes that no “standard” composition theorem is needed because a single global noise scale is calibrated against all user budgets. The utility analysis is minimax. Defining \[ H(\epsilon_1,\ldots,\epsilon_n)\coloneqq \max_{0\le k\le n}\frac{1}{\left(\sum_{i=1}^k \epsilon_i\right)^2+(n-k)}, \] the minimax mean-squared error satisfies \[ c_1\,[H(\boldsymbol\epsilon)\wedge \tfrac14]\le L(\boldsymbol\epsilon)\le c_2\,[H(\boldsymbol\epsilon)\wedge \tfrac14] \] for universal constants $c_1,c_2>0$, and ADPM achieves the upper bound with $c_2=1$ up to small constant factors. The optimization underlying the upper bound is \[ \min_{w\in \Delta_n}\left\{\frac12\|w\|_2^2+2\|w/\epsilon\|_\infty^2\right\}, \] with a water-filling argument, quasi-convexity, and KKT conditions yielding the recursion implemented by ADPM. A central structural phenomenon is saturation. If $k^*$ is the smallest index such that \[ \epsilon_{k^*+1}\ge \frac{2\sum_{i=1}^{k^*}r_i^2+8}{4\sum_{i=1}^{k^*}r_i}, \] then for $i\le k^*$ one has $r_i=\epsilon_i$, while for $i>k^*$ the recursion saturates and all $r_i$ become equal to the same common value independent of $\epsilon_i$. The paper interprets this as a “privacy-for-free” effect: once the most stringent users determine the error rate, increasing the budgets of less stringent users does not further improve the minimax rate [2310.13137]. ## 4. Correlation-aware AHDP and its operational meaning "Managing Correlations in Data and Privacy Demand" argues that conventional HDP can fail when user data and privacy demand are correlated [2509.02856]. In that setting, holding $\epsilon_i$ fixed while varying only $x_i$ is insufficient because the pair $(x_i,\epsilon_i)$ may itself be private and correlated. The proposed remedy is a correlation-aware AHDP framework in which each tuple $(x,\epsilon)$ is treated as an atomic type and neighboring changes are measured on multisets of such pairs. Formally, with dataset $D\in \mathcal S(\mathcal W)$ and count function $h_D(w)$, the weight function $\alpha:\mathcal W\to \mathbb R_{\ge 0}$ is required later to satisfy $\alpha(x,\epsilon)\le \epsilon$ for all $(x,\epsilon)\in\mathcal W$. The add–remove distance is \[ d_\alpha(D,D')=\sum_{w\in\mathcal W}\alpha(w)\,|h_D(w)-h_{D'}(w)|. \] A randomized mechanism is $\alpha$-AHDP if \[ \left|\log \frac{\Pr[M(D)\in S]}{\Pr[M(D')\in S]}\right|\le d_\alpha(D,D') \] for every $D,D'$ and measurable $S\subseteq \mathcal Y$. If $\mathcal W=\mathcal X\times\{\epsilon\}$ is homogeneous and $\alpha\equiv \epsilon$, this recovers add–remove $\epsilon$-DP. The summary also states that post-processing and composition hold exactly as in ordinary DP, with $d_{\alpha_1}+d_{\alpha_2}=d_{\alpha_1+\alpha_2}$. The paper supplies an operational interpretation through binary hypothesis testing. Given observations $Y=M(D)$ and hypotheses \[ H_0:\; Y\sim M(D),\qquad H_1:\; Y\sim M(D'), \] for a rejection region $R\subseteq \mathcal Y$ define \[ e_1(R;D,D')=\Pr_{Y\sim M(D)}[Y\in R],\qquad e_2(R;D,D')=\Pr_{Y\sim M(D')}[Y\notin R]. \] Then $M$ is $\alpha$-AHDP if and only if \[ e_1(R;D,D')+e^{\,d_\alpha(D,D')}\,e_2(R;D,D')\ge 1. \] The summary further states that, in AHDP, if an adversary might be distinguishing $D$ from $D+(x,\epsilon)$, her power is at most $1/(1+e^{-\epsilon})$, and that for a sequence of threat models with $t\to\infty$ unseen points one obtains \[ \lim_{t\to\infty}\mathcal P(M,\mathcal H_t)\le \prod_{x\in \mathcal X}\bigl(1-e^{-\epsilon_\ell(x)}\bigr), \] where $\epsilon_\ell(x)=\min\{\epsilon:(x,\epsilon)\in \mathcal W\}$. A major contrast with standard HDP is explicit: the summary states that standard HDP can completely fail under correlations, and that one can leak the entire dataset in some contrived but valid HDP mechanism. The AHDP reformulation is presented as robust to such correlations because the privacy guarantee is indexed by the combined data–privacy type rather than data alone [2509.02856]. ## 5. Universal mechanisms under correlation-aware AHDP The correlation-aware paper develops universal AHDP mechanisms, meaning mechanisms that are $\mathcal W$-AHDP for every finite $\mathcal W\subset \mathcal X\times \mathbb R_{\ge 0}$ as soon as the chosen weights satisfy $\alpha(x,\epsilon)\le \epsilon$ [2509.02856]. The summary gives four concrete families. The first is a Sampling Mechanism, attributed to Jørgensen et al. ’15. Given dataset $D\subseteq \mathcal W$, thresholds $\alpha(\cdot)\le \epsilon(\cdot)$, and $t>0$, each $(x,\epsilon)\in D$ is included in a subsample $D'$ independently with probability $p(x,\epsilon)$, after which any homogeneous $t$-DP algorithm is run on $D'$. The resulting mechanism is stated to be $(\alpha\wedge t)$-AHDP. The second family covers linear-counting queries such as sums, counts, and histograms. For $f:\mathcal X\to \mathbb R$ with $\ell\le f(x)\le h$ and \[ T(D)=\{(x,\epsilon): h_D(x,\epsilon)>0\}, \] the mechanism is \[ M(D)=\ell+\sum_{(x,\epsilon)\in T(D)}\alpha(x,\epsilon)\,(f(x)-\ell)\,h_D(x,\epsilon)+\Lap(h-\ell). \] Its add–remove sensitivity in the weighted metric is exactly $h-\ell$, which justifies the Laplace scale, and any choice $\alpha\le \epsilon$ yields universal AHDP. The third family is mean estimation via two queries for $\mathcal X\subset[\ell,h]$. With two weight functions $\alpha_1,\alpha_2\le \epsilon$, the numerator and denominator are \[ N(D)=\ell+\sum \alpha_1(x,\epsilon)(x-\ell)+\Lap(h-\ell), \] and \[ S(D)=\sum \alpha_2(x,\epsilon)+\Lap(1), \] and the output is \[ \hat\mu(D)=\frac{N(D)}{\max\{S(D),1\}}. \] By composition, $N$ is $\alpha_1$-AHDP and $S$ is $\alpha_2$-AHDP, so $\hat\mu$ is $(\alpha_1+\alpha_2)$-AHDP. The fourth family is linear regression. For data \[ D=\{(x_i,y_i,\epsilon_i)\}_{i=1}^n,\qquad x_i\in[-1,1]^d,\ y_i\in[-1,1], \] choose weights $w_i=\alpha(x_i,y_i,\epsilon_i)\le \epsilon_i$, form $X\in\mathbb R^{n\times d}$, $y\in\mathbb R^n$, and $W=\mathrm{diag}(w_1,\ldots,w_n)$, then compute \[ A=X^\top W X+\Lap\!\left(\frac{1}{d^2+d}\right)^{d\times d},\qquad b=X^\top W y+\Lap\!\left(\frac{1}{d^2+d}\right)^d, \] and output $\hat\theta=A^{-1}b$. The summary states that each entry of $A$ and $b$ has add–remove sensitivity $1/(d^2+d)$ in $w_i$, so the Laplace scale yields $\alpha$-AHDP. The empirical section evaluates these mechanisms on two GPT-4o-generated synthetic datasets and one real dataset. For mean estimation, the synthetic data contain approximately $2\,200$ samples of $(\text{body weight},\epsilon\in[0,3])$ with correlation approximately $-0.84$; Sampling Mechanism $(t=0.5)$ yields $\mathrm{MSE}\to 0$ as $n\to\infty$, whereas fixed-weight methods incur a constant bias. For frequency estimation, the data contain $3\,000$ GPT-generated tuples across $6$ education levels and $5$ $\epsilon$-levels, with a $\chi^2$ test rejecting independence; Sampling $(t=0.1)$ outperforms but does not vanish as $n\to\infty$, while linear-query methods have nontrivial error. For linear regression on California Housing, with $18\,000$ training and $2\,000$ test samples and $\epsilon\sim \log\text{-Uniform}[10^{-5},10^2]$, weighted least squares with $\alpha=\epsilon/2$ has the lowest error among universal AHDP schemes and approaches the nonprivate baseline as $n$ grows. The same summary reports computational costs of $O(n)$ for sums and histograms and $O(nd^2+d^3)$ for regression. ## 6. Conceptual issues, relationships, and open directions Several technical issues recur across the AHDP literature. One is the relation to personalized DP. The graph-based paper states that allowing $\varepsilon$ to be a function of neighboring datasets recovers an earlier definition of personalized DP as a special case [2203.15429]. Another is the status of correlation between data and privacy demand. The correlation-aware paper makes this a central objection to standard HDP, asserting that frameworks which vary data while holding privacy labels fixed can be inadequate when $(x_i,\epsilon_i)$ is correlated and private [2509.02856]. A common misconception is that heterogeneity merely means replacing a single $\epsilon$ with a vector $(\epsilon_1,\ldots,\epsilon_n)$ and then applying standard intuitions unchanged. The papers collectively indicate that this is incomplete. In the graph formulation, heterogeneity propagates through paths and induced constraints, so compatibility and optimal extension are global graph problems rather than local budget substitutions. In mean estimation, the minimax rate is controlled by \[ H(\epsilon_1,\ldots,\epsilon_n)=\max_{0\le k\le n}\frac{1}{\left(\sum_{i=1}^k \epsilon_i\right)^2+(n-k)}, \] which encodes a prefix effect rather than simple averaging of budgets. In the correlation-aware formulation, the relevant protected object is the joint type $(x,\epsilon)$, not merely the datum $x$. The papers also identify distinct bias–variance trade-offs. In graph-based AHDP, utility is framed as maximizing $\Pr[\mathcal M(v)=T(v)]$ at each vertex subject to heterogeneous DP, and the extension is pointwise optimal. In the mean-estimation setting, ADPM balances $\|w\|_2^2$ against $\|w/\epsilon\|_\infty^2$, yielding the saturation phenomenon in which larger-budget users may receive more privacy than requested without utility loss [2310.13137]. In universal AHDP mechanisms for correlated data, linear-query methods trade bias from down-weighting private points against DP noise, while sampling trades subsampling bias against DP noise; the summary explicitly notes that the choice of $t$ or $\alpha(\cdot)$ must be tuned [2509.02856]. Open problems are stated in task-specific terms. For heterogeneous mean estimation these include tighter heterogeneous-DP composition theorems, data-dependent tuning of weights when $\epsilon_i$ are private, extensions to sub-Gaussian or heavy-tailed domains, multivariate AHDP, and practical implementations in streaming or federated settings [2310.13137]. For correlation-aware AHDP, the listed directions include $(\epsilon,\delta)$-AHDP or Rényi-AHDP variants via Gaussian mechanisms, task-specific mechanisms beyond linear models, membership-inference stress tests, characterization of minimal bias under realistic distributional assumptions, and optimal budget splitting between numerator and denominator in mean estimation [2509.02856]. Taken together, these works present AHDP as a technically heterogeneous research area organized around a shared principle: add/remove privacy guarantees can be individualized, but the correct formal object for that individualization depends on whether the heterogeneity is attached to graph edges, user coordinates, or joint data–privacy types. This suggests that the central unresolved question is not whether heterogeneous privacy is possible, but which add/remove formalism is appropriate for the statistical, structural, and adversarial assumptions of a given problem. Source: https://www.emergentmind.com/topics/add-remove-heterogeneous-differential-privacy-ahdp