---
title: 'PACE-GGM: Adaptive DP Covariance Estimation'
url: https://www.emergentmind.com/topics/pace-ggm
type: topic
---

# PACE-GGM: Adaptive DP Covariance Estimation

PACE-GGM, short for **Private Adaptive Covariance Estimation via Gaussian Graphical Models**, is a data-adaptive method for differentially private covariance estimation in the central model of differential privacy. It estimates the empirical second-moment matrix
\[
\Sigma(X)=\frac{1}{n}X^\top X
\]
for centered data by avoiding uniform perturbation of all \(d(d+1)/2\) covariance entries. Instead, it privately selects covariance entries that are currently poorly approximated, privately measures only those entries, and reconstructs a full positive semidefinite covariance matrix by a maximum-entropy objective whose solution induces a Gaussian graphical model structure [2605.24295].

## 1. Statistical setting and motivation

PACE-GGM is formulated for datasets \(X\in\mathbb{R}^{n\times d}\) with rows \(x^{(1)},\dots,x^{(n)}\), under the assumption that the data are centered so that \(\Sigma(X)\) is also the empirical covariance matrix. Its motivation is a limitation of standard private covariance release methods such as sufficient statistic perturbation (SSP): they apply the Gaussian mechanism to the entire covariance matrix, perturbing all entries at once regardless of which entries are informative and regardless of whether structural assumptions could recover many of them indirectly [2605.24295].

The method is tailored to the setting in which the modeler supplies **separate bounds for each variable**, rescaled to a common coordinate-wise bound
\[
|x_j|\le B \quad \text{for all coordinates } j,
\]
equivalently \(\|x\|_\infty\le B\). Under this assumption, the sensitivity of a **single covariance entry** is
\[
\Delta=\Delta_{\Sigma_{jk}}=\frac{2B^2}{n},
\]
whereas the sensitivity of the **full covariance matrix** is
\[
\Delta_\Sigma=\frac{\sqrt{2}\,dB^2}{n}.
\]
This gap is the central design premise of PACE-GGM: for fixed privacy budget, selected entries can be measured with much less noise than a full-matrix release. The paper contrasts this with the \(\ell_2\)-bounded regime \(\|x\|_2\le C\), where the full-matrix sensitivity \(\frac{\sqrt{2}C^2}{n}\) and single-entry sensitivity \(\frac{C^2}{n}\) differ only by a constant factor, making adaptive entry selection less compelling [2605.24295].

The resulting intuition is explicitly structural. When \(d\) is large, privacy is limited, and covariance structure is sparse or otherwise recoverable from a relatively small set of informative entries, uniform perturbation wastes privacy budget. PACE-GGM therefore concentrates budget on entries that are poorly approximated by the current estimate, then uses a structured PSD reconstruction to infer the remainder. The paper characterizes this as especially beneficial in **high-dimensional** and **low-to-moderate privacy** regimes [2605.24295].

## 2. Privacy model and sensitivity calculus

PACE-GGM is analyzed under **zero-concentrated differential privacy (zCDP)** in the **bounded DP model**. Two datasets \(X,X'\in\mathcal X^n\) are neighboring if they differ in a single record. Privacy accounting relies on the zCDP forms of both the Gaussian mechanism and the exponential mechanism, together with adaptive composition [2605.24295].

For a function \(f\) with \(\ell_2\) sensitivity \(\Delta(f)\), the Gaussian mechanism
\[
\mathcal M(X)=f(X)+\mathcal N(0,\sigma^2 I)
\]
satisfies
\[
\frac{\Delta(f)^2}{2\sigma^2}\text{-zCDP}.
\]
Accordingly, spending privacy \(\rho_0\) on one scalar covariance entry with sensitivity \(\Delta\) uses noise variance
\[
\sigma^2=\frac{\Delta^2}{2\rho_0}.
\]

Entry selection is privatized by the exponential mechanism. If the score sensitivity is \(\Delta\), then selecting \(r\) with probability proportional to
\[
\exp\!\left(\frac{\epsilon}{2\Delta}\text{score}_r(X)\right)
\]
satisfies
\[
\frac{\epsilon^2}{8}\text{-zCDP}.
\]
PACE-GGM instantiates this by setting
\[
\epsilon=\sqrt{8\rho_{\text{sel}}},
\]
with roundwise selection score
\[
\text{err}(j,k)=\left|\Sigma_{jk}(X)-\hat\Sigma^{(t-1)}_{jk}\right|.
\]
Thus the algorithm spends privacy to favor entries whose current reconstruction error is large, while keeping the selection itself private [2605.24295].

The total guarantee is additive under adaptive composition. If \(\rho_{\text{diag}}=\alpha\rho\) is used for diagonal initialization and the remaining rounds consume \(\rho_{\text{sel}}^{(t)}+\rho_{\text{meas}}^{(t)}\) with cumulative sum at most \((1-\alpha)\rho\), then the whole algorithm satisfies exactly \(\rho\)-zCDP. The paper also states the standard conversion: if \(\mathcal M\) is \(\rho\)-zCDP, then it is
\[
\bigl(\rho + 2\sqrt{\rho\ln(1/\delta)},\,\delta\bigr)\text{-DP}.
\]
This theorem is exact for privacy and does not depend on the reconstruction stage, which is pure post-processing [2605.24295].

## 3. Adaptive entry selection and measurement procedure

PACE-GGM is a **select–measure–reconstruct** estimator. Its inputs are the data \(X\in\mathbb R^{n\times d}\), privacy budget \(\rho\), coordinate bound \(B\), maximum rounds \(T\), and hyperparameters \(\alpha\in(0,1)\) and \(\beta\in(0,1)\). The defaults used in the experiments are
\[
\alpha=0.3,\qquad \beta=0.5.
\]
The maintained state consists of a current covariance estimate \(\hat\Sigma^{(t)}\), together with, for each lower-triangular entry \(j\ge k\), a precision-weighted average \(y_{jk}\) of noisy measurements and its accumulated precision \(\lambda_{jk}\) [2605.24295].

Initialization spends \(\rho_{\text{diag}}=\alpha\rho\) on all diagonal entries. With per-entry sensitivity
\[
\Delta=\frac{2B^2}{n},
\]
the diagonal variance is
\[
\sigma^2=\frac{\Delta^2}{2(\rho_{\text{diag}}/d)}.
\]
For each \(j=1,\dots,d\), the algorithm releases
\[
y_{jj}\gets \Sigma_{jj}(X)+\mathcal N(0,\sigma^2),\qquad \lambda_{jj}\gets \sigma^{-2},
\]
and forms the initial estimate
\[
\hat\Sigma^{(0)}\gets \operatorname{diag}(y_{11}\vee 0,\dots,y_{dd}\vee 0).
\]
Thus the procedure begins from a private diagonal covariance estimate [2605.24295].

After initialization, the remaining budget is divided into rounds. The initial per-round allocation is
\[
\rho_{\text{per-round}}\gets \frac{\rho-\rho_{\text{used}}}{T},
\]
with split
\[
\rho_{\text{sel}}\gets \beta\,\rho_{\text{per-round}},\qquad
\rho_{\text{meas}}\gets (1-\beta)\,\rho_{\text{per-round}}.
\]
At round \(t\), the algorithm first selects an entry \((j,k)\) using the exponential mechanism with score \(\text{err}(j,k)\). It then privately measures that entry by the Gaussian mechanism:
\[
z_{jk}\gets \Sigma_{jk}(X)+\mathcal N(0,\sigma_t^2),\qquad
\sigma_t^2=\frac{\Delta^2}{2\rho_{\text{meas}}}.
\]

Repeated measurements of the same entry are combined by inverse-variance weighting:
\[
y_{jk}\gets
\frac{\lambda_{jk}y_{jk}+\sigma_t^{-2}z_{jk}}
{\lambda_{jk}+\sigma_t^{-2}},
\qquad
\lambda_{jk}\gets \lambda_{jk}+\sigma_t^{-2}.
\]
This aggregation is exact for Gaussian observations and compresses repeated measurements to sufficient statistics \((y_{jk},\lambda_{jk})\) [2605.24295].

PACE-GGM also includes a budget-annealing rule. After reconstruction, if the newly measured entry causes only a small update,
\[
|\hat{\Sigma}_{jk}^{(t)}-\hat{\Sigma}_{jk}^{(t-1)}|
\le
\sqrt{\frac{2}{\pi}}\,\sigma_t,
\]
then future rounds become more expensive but more informative:
\[
\rho_{\text{sel}}\gets 2\rho_{\text{sel}},\qquad
\rho_{\text{meas}}\gets 4\rho_{\text{meas}}.
\]
The loop continues while \(\rho_{\text{used}}<\rho\). In the experiments, the upper bound on rounds is
\[
T=\left\lfloor 2\cdot \frac{d(d-1)}{2}\right\rfloor,
\]
and the procedure is allowed to remeasure entries, including diagonal entries [2605.24295].

## 4. Maximum-entropy reconstruction and Gaussian graphical model structure

The reconstruction stage is the defining feature behind the “GGM” in PACE-GGM. Given the currently measured set \(S=\{(j,k):\lambda_{jk}>0\}\), with noisy observations modeled as
\[
y_{jk}=\Sigma_{jk}+\mathcal N(0,\tau_{jk}^2),\qquad (j,k)\in S,
\]
where \(\tau_{jk}^2=\lambda_{jk}^{-1}\), the weighted least-squares fitting loss is
\[
\mathcal L(W)=\sum_{(j,k)\in S}\lambda_{jk}(W_{jk}-y_{jk})^2.
\]
PACE-GGM then reconstructs a full covariance matrix by solving the bilevel maximum-entropy problem
\[
\hat{\Sigma}^{(t)}
\gets
\argmax\left\{\log|\Sigma| :
\Sigma \in \arg\min_{W \succeq 0}
\sum_{j \ge k} \lambda_{jk}(W_{jk} - y_{jk})^2
\right\}.
\]
The inner problem fits the measured entries subject to PSD structure; the outer problem selects, among equally good PSD fits, the one with largest log determinant. The paper interprets this as a maximum-entropy PSD completion and relates it directly to covariance selection in the sense of Dempster [2605.24295].

This maximum-entropy principle produces the Gaussian graphical model interpretation. If
\[
K^*=(\Sigma^*)^{-1},
\]
then the paper proves that if the minimizers of \(\mathcal L\) include a positive definite matrix, the reconstruction has a unique solution \(\Sigma^*\) and
\[
K^*_{jk}=0
\quad \text{for } (j,k),(k,j)\notin S.
\]
Accordingly, measured entries define the graph, while unmeasured pairs induce zeros in the precision matrix. The completed covariance is therefore the covariance of a Gaussian graphical model consistent with those sparsity constraints [2605.24295].

Because direct solution of the bilevel problem can be numerically unstable, the implementation uses a barrier formulation. For \(\mu>0\),
\[
\Sigma_\mu^*=
\arg\min_{W \succeq 0}\left(\mathcal L(W)-\mu\log|W|\right),
\]
and as \(\mu\to 0\), \(\Sigma_\mu^*\) converges to the desired maximum-entropy limit when the problem is well posed. Using a Cholesky factorization \(\Sigma=LL^\top\), the optimized objective becomes
\[
\varphi_\mu(L)=
\sum_{(j,k)\in S}
\frac{((LL^\top)_{jk}-y_{jk})^2}{2\tau_{jk}^2}
-
2\mu\sum_{j=1}^d \log L_{jj},
\]
with constraints \(L_{jj}\ge 0\). The solver is an interior-point-style barrier method with Cholesky parameterization, termed **IPM-Cholesky**, and optimized using **L-BFGS-B** [2605.24295].

The paper also exploits graph sparsity computationally. If the measured entries induce a graph with connected components \(V_1,\dots,V_K\), then reconstruction decomposes across blocks, reducing cost from \(O(d^3)\) to
\[
O\!\left(\sum_i d_i^3\right)
\]
for component sizes \(d_i\). This decomposition is exact and is one reason the method can remain practical even when very few off-diagonal entries are measured [2605.24295].

## 5. Empirical evaluation and observed operating regimes

PACE-GGM is evaluated on **14 real-world tabular datasets** with dimensions ranging from
\[
d=6 \text{ to } d=260.
\]
Examples listed in the paper include Adult (\(d=6\)), SeoulBike (\(d=10\)), LifeExpectancy (\(d=15\)), thyroid\_ann (\(d=21\)), ibm (\(d=23\)), bank32nh (\(d=32\)), BreastCancerWisconsin (\(d=32\)), spambase (\(d=57\)), communities\_crime (\(d=102\)), indian\_pines (\(d=187\)), and madeline (\(d=260\)). Preprocessing removes rows with missing values, rescales each feature to \([-1,1]\) so that \(B=1\), and centers the data using the known mean to isolate covariance estimation [2605.24295].

The privacy budgets are
\[
\rho\in\{10^{-4},10^{-3},10^{-2},10^{-1},1,2,10\},
\]
with \(10\) trials per configuration. The reported error metrics are the **Mahalanobis error**
\[
\|\Sigma^{-1/2}\hat{\Sigma}\Sigma^{-1/2} - I\|_F
\]
and the **Frobenius error**
\[
\|\hat{\Sigma} - \Sigma\|_F.
\]
The main baselines are SSP, AdaptiveCov, and a diagonal-only baseline that privatizes only the diagonal and sets off-diagonals to zero before PSD projection [2605.24295].

The paper reports that PACE-GGM **overall outperforms SSP** and **often also beats AdaptiveCov** on both Mahalanobis and Frobenius error. The gains are strongest when **dimension is large**, **privacy budget is low to moderate**, or the covariance structure is such that only a sparse subset of entries needs direct measurement. Conversely, the diagonal-only baseline is competitive only on datasets with nearly diagonal covariance, such as Adult and thyroid\_ann, where off-diagonal energy is tiny [2605.24295].

A distinctive empirical observation is the sparsity of direct measurement. At \(\rho=10\), Adult (\(d=6\)) uses about \(12\) off-diagonal measurements out of \(15\), whereas madeline (\(d=260\)) uses about \(39\) off-diagonal measurements out of \(33{,}930\), roughly \(0.1\%\). On communities\_crime (\(d=102\)), whose covariance is denser, the method measures about \(255\) off-diagonal entries. This behavior indicates that the method adapts to covariance density rather than dimension alone [2605.24295].

Hyperparameter ablations vary \(\alpha,\beta\in\{0.1,0.3,0.5\}\). The reported conclusion is that larger \(\alpha\) can help at very low privacy budgets because stronger diagonal initialization matters more, while \(\beta=0.5\) often works best, suggesting that selection quality is important. The default \((\alpha,\beta)=(0.3,0.5)\) is reported as a good general choice. Solver ablations further report that IPM-Cholesky is more stable than projected gradient descent, and exploiting connected components preserves accuracy while reducing runtime [2605.24295].

Runtime behavior is heterogeneous. Many datasets with \(d\le 57\) run in under 3 seconds even at \(\rho=10\), whereas communities\_crime can take tens of seconds and indian\_pines can take thousands of seconds at high \(\rho\). The paper notes that higher dimension does not always imply slower runtime: madeline remains relatively fast because its covariance is close to diagonal, so few entries are measured [2605.24295].

## 6. Naming, scope, and limitations

The name **PACE-GGM** is specific to **Private Adaptive Covariance Estimation via Gaussian Graphical Models** and should be distinguished from several unrelated arXiv papers using the acronym “PACE” for different topics, including robot policy execution [2606.00537], acceptance tests for self-evolving agents [2606.08106], battery health estimation [2512.11332], single-cell trajectory inference [2605.18587], and incomplete-information differential games [2504.17128]. In those works, “GGM” does not appear; the covariance-estimation paper is the one that explicitly defines PACE-GGM [2605.24295].

Within private statistics, the paper presents PACE-GGM as an adaptive covariance estimator with seven prominent characteristics: selective entry measurement, use of coordinate-wise bounded data, PSD reconstruction, a Gaussian graphical model interpretation, clean handling of repeated measurements via inverse-variance weighting, exact \(\rho\)-zCDP accounting, and strong empirical performance especially in high-dimensional and low-to-moderate privacy regimes [2605.24295].

The stated limitations are equally important. The method assumes known coordinate-wise bounds \(\|x\|_\infty\le B\), supplied through domain knowledge or clipping. The experiments also assume the mean is known, so practical deployment would require spending additional privacy budget on mean estimation. The Gaussian graphical model reconstruction may introduce bias when the covariance does not align well with the induced precision sparsity. The paper proves privacy, but it does **not** provide a full theoretical utility guarantee for the adaptive estimator. Computational overhead can become substantial when \(d\) is large and many correlations remain informative. Heavy-tailed or strongly non-Gaussian data were not evaluated, and no private hyperparameter tuning procedure is given [2605.24295].

The paper also specifies when adaptivity may be less useful. PACE-GGM may be less advantageous when a tight global \(\ell_2\) bound is already available, when the covariance is nearly diagonal so that diagonal-only release suffices, when the privacy budget is very large and full-matrix perturbation becomes less problematic, or when computation is the primary bottleneck. A plausible implication is that PACE-GGM is best viewed not as a universal replacement for SSP, but as a method optimized for the regime in which entrywise covariance queries are much cheaper than releasing the full covariance matrix and where structural completion can exploit that asymmetry [2605.24295].

Source: https://www.emergentmind.com/topics/pace-ggm