PACE-GGM: Adaptive DP Covariance Estimation

• PACE-GGM is a method for differentially private covariance estimation that selectively measures informative covariance entries rather than applying uniform noise.
• It employs a maximum-entropy reconstruction to complete a positive semidefinite matrix, inducing Gaussian graphical model sparsity for unmeasured pairs.
• Empirical results show that PACE-GGM improves accuracy over traditional methods, especially in high-dimensional settings and low-to-moderate privacy regimes.

PACE-GGM, short for Private Adaptive Covariance Estimation via Gaussian Graphical Models, is a data-adaptive method for differentially private covariance estimation in the central model of differential privacy. It estimates the empirical second-moment matrix

$\Sigma(X)=\frac{1}{n}X^\top X$

for centered data by avoiding uniform perturbation of all $d(d+1)/2$ covariance entries. Instead, it privately selects covariance entries that are currently poorly approximated, privately measures only those entries, and reconstructs a full positive semidefinite covariance matrix by a maximum-entropy objective whose solution induces a Gaussian graphical model structure (Ferrando et al., 22 May 2026).

1. Statistical setting and motivation

PACE-GGM is formulated for datasets $X\in\mathbb{R}^{n\times d}$ with rows $x^{(1)},\dots,x^{(n)}$, under the assumption that the data are centered so that $\Sigma(X)$ is also the empirical covariance matrix. Its motivation is a limitation of standard private covariance release methods such as sufficient statistic perturbation (SSP): they apply the Gaussian mechanism to the entire covariance matrix, perturbing all entries at once regardless of which entries are informative and regardless of whether structural assumptions could recover many of them indirectly (Ferrando et al., 22 May 2026).

The method is tailored to the setting in which the modeler supplies separate bounds for each variable, rescaled to a common coordinate-wise bound

$|x_j|\le B \quad \text{for all coordinates } j,$

equivalently $\|x\|_\infty\le B$. Under this assumption, the sensitivity of a single covariance entry is

$\Delta=\Delta_{\Sigma_{jk}}=\frac{2B^2}{n},$

whereas the sensitivity of the full covariance matrix is

$\Delta_\Sigma=\frac{\sqrt{2}\,dB^2}{n}.$

This gap is the central design premise of PACE-GGM: for fixed privacy budget, selected entries can be measured with much less noise than a full-matrix release. The paper contrasts this with the $\ell_2$-bounded regime $d(d+1)/2$0, where the full-matrix sensitivity $d(d+1)/2$1 and single-entry sensitivity $d(d+1)/2$2 differ only by a constant factor, making adaptive entry selection less compelling (Ferrando et al., 22 May 2026).

The resulting intuition is explicitly structural. When $d(d+1)/2$3 is large, privacy is limited, and covariance structure is sparse or otherwise recoverable from a relatively small set of informative entries, uniform perturbation wastes privacy budget. PACE-GGM therefore concentrates budget on entries that are poorly approximated by the current estimate, then uses a structured PSD reconstruction to infer the remainder. The paper characterizes this as especially beneficial in high-dimensional and low-to-moderate privacy regimes (Ferrando et al., 22 May 2026).

2. Privacy model and sensitivity calculus

PACE-GGM is analyzed under zero-concentrated differential privacy (zCDP) in the bounded DP model. Two datasets $d(d+1)/2$4 are neighboring if they differ in a single record. Privacy accounting relies on the zCDP forms of both the Gaussian mechanism and the exponential mechanism, together with adaptive composition (Ferrando et al., 22 May 2026).

For a function $d(d+1)/2$5 with $d(d+1)/2$6 sensitivity $d(d+1)/2$7, the Gaussian mechanism

$d(d+1)/2$8

satisfies

$d(d+1)/2$9

Accordingly, spending privacy $X\in\mathbb{R}^{n\times d}$0 on one scalar covariance entry with sensitivity $X\in\mathbb{R}^{n\times d}$1 uses noise variance

$X\in\mathbb{R}^{n\times d}$2

Entry selection is privatized by the exponential mechanism. If the score sensitivity is $X\in\mathbb{R}^{n\times d}$3, then selecting $X\in\mathbb{R}^{n\times d}$4 with probability proportional to

$X\in\mathbb{R}^{n\times d}$5

satisfies

$X\in\mathbb{R}^{n\times d}$6

PACE-GGM instantiates this by setting

$X\in\mathbb{R}^{n\times d}$7

with roundwise selection score

$X\in\mathbb{R}^{n\times d}$8

Thus the algorithm spends privacy to favor entries whose current reconstruction error is large, while keeping the selection itself private (Ferrando et al., 22 May 2026).

The total guarantee is additive under adaptive composition. If $X\in\mathbb{R}^{n\times d}$9 is used for diagonal initialization and the remaining rounds consume $x^{(1)},\dots,x^{(n)}$0 with cumulative sum at most $x^{(1)},\dots,x^{(n)}$1, then the whole algorithm satisfies exactly $x^{(1)},\dots,x^{(n)}$2-zCDP. The paper also states the standard conversion: if $x^{(1)},\dots,x^{(n)}$3 is $x^{(1)},\dots,x^{(n)}$4-zCDP, then it is

$x^{(1)},\dots,x^{(n)}$5

This theorem is exact for privacy and does not depend on the reconstruction stage, which is pure post-processing (Ferrando et al., 22 May 2026).

3. Adaptive entry selection and measurement procedure

PACE-GGM is a select–measure–reconstruct estimator. Its inputs are the data $x^{(1)},\dots,x^{(n)}$6, privacy budget $x^{(1)},\dots,x^{(n)}$7, coordinate bound $x^{(1)},\dots,x^{(n)}$8, maximum rounds $x^{(1)},\dots,x^{(n)}$9, and hyperparameters $\Sigma(X)$0 and $\Sigma(X)$1. The defaults used in the experiments are

$\Sigma(X)$2

The maintained state consists of a current covariance estimate $\Sigma(X)$3, together with, for each lower-triangular entry $\Sigma(X)$4, a precision-weighted average $\Sigma(X)$5 of noisy measurements and its accumulated precision $\Sigma(X)$6 (Ferrando et al., 22 May 2026).

Initialization spends $\Sigma(X)$7 on all diagonal entries. With per-entry sensitivity

$\Sigma(X)$8

the diagonal variance is

$\Sigma(X)$9

For each $|x_j|\le B \quad \text{for all coordinates } j,$0, the algorithm releases

$|x_j|\le B \quad \text{for all coordinates } j,$1

and forms the initial estimate

$|x_j|\le B \quad \text{for all coordinates } j,$2

Thus the procedure begins from a private diagonal covariance estimate (Ferrando et al., 22 May 2026).

After initialization, the remaining budget is divided into rounds. The initial per-round allocation is

$|x_j|\le B \quad \text{for all coordinates } j,$3

with split

$|x_j|\le B \quad \text{for all coordinates } j,$4

At round $|x_j|\le B \quad \text{for all coordinates } j,$5, the algorithm first selects an entry $|x_j|\le B \quad \text{for all coordinates } j,$6 using the exponential mechanism with score $|x_j|\le B \quad \text{for all coordinates } j,$7. It then privately measures that entry by the Gaussian mechanism: $|x_j|\le B \quad \text{for all coordinates } j,$8

Repeated measurements of the same entry are combined by inverse-variance weighting: $|x_j|\le B \quad \text{for all coordinates } j,$9 This aggregation is exact for Gaussian observations and compresses repeated measurements to sufficient statistics $\|x\|_\infty\le B$0 (Ferrando et al., 22 May 2026).

PACE-GGM also includes a budget-annealing rule. After reconstruction, if the newly measured entry causes only a small update,

$\|x\|_\infty\le B$1

then future rounds become more expensive but more informative: $\|x\|_\infty\le B$2 The loop continues while $\|x\|_\infty\le B$3. In the experiments, the upper bound on rounds is

$\|x\|_\infty\le B$4

and the procedure is allowed to remeasure entries, including diagonal entries (Ferrando et al., 22 May 2026).

4. Maximum-entropy reconstruction and Gaussian graphical model structure

The reconstruction stage is the defining feature behind the “GGM” in PACE-GGM. Given the currently measured set $\|x\|_\infty\le B$5, with noisy observations modeled as

$\|x\|_\infty\le B$6

where $\|x\|_\infty\le B$7, the weighted least-squares fitting loss is

$\|x\|_\infty\le B$8

PACE-GGM then reconstructs a full covariance matrix by solving the bilevel maximum-entropy problem

$\|x\|_\infty\le B$9

The inner problem fits the measured entries subject to PSD structure; the outer problem selects, among equally good PSD fits, the one with largest log determinant. The paper interprets this as a maximum-entropy PSD completion and relates it directly to covariance selection in the sense of Dempster (Ferrando et al., 22 May 2026).

This maximum-entropy principle produces the Gaussian graphical model interpretation. If

$\Delta=\Delta_{\Sigma_{jk}}=\frac{2B^2}{n},$0

then the paper proves that if the minimizers of $\Delta=\Delta_{\Sigma_{jk}}=\frac{2B^2}{n},$1 include a positive definite matrix, the reconstruction has a unique solution $\Delta=\Delta_{\Sigma_{jk}}=\frac{2B^2}{n},$2 and

$\Delta=\Delta_{\Sigma_{jk}}=\frac{2B^2}{n},$3

Accordingly, measured entries define the graph, while unmeasured pairs induce zeros in the precision matrix. The completed covariance is therefore the covariance of a Gaussian graphical model consistent with those sparsity constraints (Ferrando et al., 22 May 2026).

Because direct solution of the bilevel problem can be numerically unstable, the implementation uses a barrier formulation. For $\Delta=\Delta_{\Sigma_{jk}}=\frac{2B^2}{n},$4,

$\Delta=\Delta_{\Sigma_{jk}}=\frac{2B^2}{n},$5

and as $\Delta=\Delta_{\Sigma_{jk}}=\frac{2B^2}{n},$6, $\Delta=\Delta_{\Sigma_{jk}}=\frac{2B^2}{n},$7 converges to the desired maximum-entropy limit when the problem is well posed. Using a Cholesky factorization $\Delta=\Delta_{\Sigma_{jk}}=\frac{2B^2}{n},$8, the optimized objective becomes

$\Delta=\Delta_{\Sigma_{jk}}=\frac{2B^2}{n},$9

with constraints $\Delta_\Sigma=\frac{\sqrt{2}\,dB^2}{n}.$0. The solver is an interior-point-style barrier method with Cholesky parameterization, termed IPM-Cholesky, and optimized using L-BFGS-B (Ferrando et al., 22 May 2026).

The paper also exploits graph sparsity computationally. If the measured entries induce a graph with connected components $\Delta_\Sigma=\frac{\sqrt{2}\,dB^2}{n}.$1, then reconstruction decomposes across blocks, reducing cost from $\Delta_\Sigma=\frac{\sqrt{2}\,dB^2}{n}.$2 to

$\Delta_\Sigma=\frac{\sqrt{2}\,dB^2}{n}.$3

for component sizes $\Delta_\Sigma=\frac{\sqrt{2}\,dB^2}{n}.$4. This decomposition is exact and is one reason the method can remain practical even when very few off-diagonal entries are measured (Ferrando et al., 22 May 2026).

5. Empirical evaluation and observed operating regimes

PACE-GGM is evaluated on 14 real-world tabular datasets with dimensions ranging from

$\Delta_\Sigma=\frac{\sqrt{2}\,dB^2}{n}.$5

Examples listed in the paper include Adult ($\Delta_\Sigma=\frac{\sqrt{2}\,dB^2}{n}.$6), SeoulBike ($\Delta_\Sigma=\frac{\sqrt{2}\,dB^2}{n}.$7), LifeExpectancy ($\Delta_\Sigma=\frac{\sqrt{2}\,dB^2}{n}.$8), thyroid_ann ($\Delta_\Sigma=\frac{\sqrt{2}\,dB^2}{n}.$9), ibm ($\ell_2$0), bank32nh ($\ell_2$1), BreastCancerWisconsin ($\ell_2$2), spambase ($\ell_2$3), communities_crime ($\ell_2$4), indian_pines ($\ell_2$5), and madeline ($\ell_2$6). Preprocessing removes rows with missing values, rescales each feature to $\ell_2$7 so that $\ell_2$8, and centers the data using the known mean to isolate covariance estimation (Ferrando et al., 22 May 2026).

The privacy budgets are

$\ell_2$9

with $d(d+1)/2$00 trials per configuration. The reported error metrics are the Mahalanobis error

$d(d+1)/2$01

and the Frobenius error

$d(d+1)/2$02

The main baselines are SSP, AdaptiveCov, and a diagonal-only baseline that privatizes only the diagonal and sets off-diagonals to zero before PSD projection (Ferrando et al., 22 May 2026).

The paper reports that PACE-GGM overall outperforms SSP and often also beats AdaptiveCov on both Mahalanobis and Frobenius error. The gains are strongest when dimension is large, privacy budget is low to moderate, or the covariance structure is such that only a sparse subset of entries needs direct measurement. Conversely, the diagonal-only baseline is competitive only on datasets with nearly diagonal covariance, such as Adult and thyroid_ann, where off-diagonal energy is tiny (Ferrando et al., 22 May 2026).

A distinctive empirical observation is the sparsity of direct measurement. At $d(d+1)/2$03, Adult ($d(d+1)/2$04) uses about $d(d+1)/2$05 off-diagonal measurements out of $d(d+1)/2$06, whereas madeline ($d(d+1)/2$07) uses about $d(d+1)/2$08 off-diagonal measurements out of $d(d+1)/2$09, roughly $d(d+1)/2$10. On communities_crime ($d(d+1)/2$11), whose covariance is denser, the method measures about $d(d+1)/2$12 off-diagonal entries. This behavior indicates that the method adapts to covariance density rather than dimension alone (Ferrando et al., 22 May 2026).

Hyperparameter ablations vary $d(d+1)/2$13. The reported conclusion is that larger $d(d+1)/2$14 can help at very low privacy budgets because stronger diagonal initialization matters more, while $d(d+1)/2$15 often works best, suggesting that selection quality is important. The default $d(d+1)/2$16 is reported as a good general choice. Solver ablations further report that IPM-Cholesky is more stable than projected gradient descent, and exploiting connected components preserves accuracy while reducing runtime (Ferrando et al., 22 May 2026).

Runtime behavior is heterogeneous. Many datasets with $d(d+1)/2$17 run in under 3 seconds even at $d(d+1)/2$18, whereas communities_crime can take tens of seconds and indian_pines can take thousands of seconds at high $d(d+1)/2$19. The paper notes that higher dimension does not always imply slower runtime: madeline remains relatively fast because its covariance is close to diagonal, so few entries are measured (Ferrando et al., 22 May 2026).

6. Naming, scope, and limitations

The name PACE-GGM is specific to Private Adaptive Covariance Estimation via Gaussian Graphical Models and should be distinguished from several unrelated arXiv papers using the acronym “PACE” for different topics, including robot policy execution (Nie et al., 30 May 2026), acceptance tests for self-evolving agents (Shawn, 6 Jun 2026), battery health estimation (Sameer et al., 12 Dec 2025), single-cell trajectory inference (Yu* et al., 18 May 2026), and incomplete-information differential games (Soltanian et al., 23 Apr 2025). In those works, “GGM” does not appear; the covariance-estimation paper is the one that explicitly defines PACE-GGM (Ferrando et al., 22 May 2026).

Within private statistics, the paper presents PACE-GGM as an adaptive covariance estimator with seven prominent characteristics: selective entry measurement, use of coordinate-wise bounded data, PSD reconstruction, a Gaussian graphical model interpretation, clean handling of repeated measurements via inverse-variance weighting, exact $d(d+1)/2$20-zCDP accounting, and strong empirical performance especially in high-dimensional and low-to-moderate privacy regimes (Ferrando et al., 22 May 2026).

The stated limitations are equally important. The method assumes known coordinate-wise bounds $d(d+1)/2$21, supplied through domain knowledge or clipping. The experiments also assume the mean is known, so practical deployment would require spending additional privacy budget on mean estimation. The Gaussian graphical model reconstruction may introduce bias when the covariance does not align well with the induced precision sparsity. The paper proves privacy, but it does not provide a full theoretical utility guarantee for the adaptive estimator. Computational overhead can become substantial when $d(d+1)/2$22 is large and many correlations remain informative. Heavy-tailed or strongly non-Gaussian data were not evaluated, and no private hyperparameter tuning procedure is given (Ferrando et al., 22 May 2026).

The paper also specifies when adaptivity may be less useful. PACE-GGM may be less advantageous when a tight global $d(d+1)/2$23 bound is already available, when the covariance is nearly diagonal so that diagonal-only release suffices, when the privacy budget is very large and full-matrix perturbation becomes less problematic, or when computation is the primary bottleneck. A plausible implication is that PACE-GGM is best viewed not as a universal replacement for SSP, but as a method optimized for the regime in which entrywise covariance queries are much cheaper than releasing the full covariance matrix and where structural completion can exploit that asymmetry (Ferrando et al., 22 May 2026).