---
title: 'AdaSSP: Adaptive Sufficient-Statistics Perturbation'
url: https://www.emergentmind.com/topics/adaptive-sufficient-statistics-perturbation-adassp
type: topic
---

# AdaSSP: Adaptive Sufficient-Statistics Perturbation

Adaptive Sufficient-Statistics Perturbation (AdaSSP) is an advanced method for differentially private (DP) linear regression that achieves near-optimal performance by calibrating privacy noise to data-dependent characteristics of the input, particularly the curvature of the sample covariance matrix. AdaSSP is characterized by its use of adaptive, instance-specific ridge regularization and data-aware noise allocation, which address the instability and suboptimality of prior sufficient-statistics perturbation (SSP) methods, especially in ill-conditioned or high-dimensional settings [2601.07545, 1803.02596, 2405.15002].

## 1. Problem Formulation and Context

The core problem addressed by AdaSSP is differentially private ordinary least squares (DP-OLS) regression. Given a design matrix $X \in \mathbb{R}^{n \times d}$ (rows $x_i$) and a response vector $Y \in \mathbb{R}^n$ (entries $y_i$), with each $\|x_i\|_2 \leq C_x$ and $|y_i| \leq C_y$, and assuming $n \geq d$ (full or near full rank), the task is to estimate
\[
\theta^* = (X^T X)^{-1} X^T Y
\]
subject to $(\epsilon, \delta)$-differential privacy. The privacy notion is defined under "zero-out" neighbors: datasets differing in a single $(x_i, y_i)$ replaced by the zero vector [2601.07545].

Classical non-adaptive SSP releases $X^T X$ and $X^T Y$ with data-independent Gaussian noise calibrated to their global sensitivities. However, if $X^T X$ is nearly singular, the estimator's variance can be arbitrarily inflated. AdaSSP addresses these failures by privately estimating spectral properties and tuning the regularization parameter to current data [1803.02596].

## 2. AdaSSP Algorithmic Structure

AdaSSP instantiates the following procedure [2601.07545, 1803.02596, 2405.15002]:

1. **Privacy budget splitting:** The total $(\epsilon, \delta)$ is divided equally ($\epsilon_1 = \epsilon_2 = \epsilon_3 = \epsilon / 3$, and likewise for $\delta$) among:
    - A private estimate of the minimum eigenvalue $\lambda_{\min}(X^T X)$.
    - Noisy release of the empirical covariance $G = X^T X$.
    - Noisy release of the cross-moment $h = X^T Y$.

2. **Private $\lambda_{\min}$ estimation:** Adds calibrated Gaussian noise to $\lambda_{\min}(G)$ (scale $\sigma_\lambda = C_x^2 \sqrt{2\log(1.25/\delta_1)}/\epsilon_1$), truncates to nonnegativity with a debiasing shift.

3. **Adaptive ridge selection:** Sets the regularization parameter $\lambda$ to
\[
\lambda = \max \left\{0, \frac{\sqrt{p \log(2p^2/\rho)} C_x^2}{\epsilon_2 / \sqrt{2\log(1.25/\delta_2)}} - \tilde{\lambda}_{\min}\right\}
\]
with $\rho$ a confidence parameter (e.g., $0.05$), balancing stabilization and minimax optimality.

4. **Noisy sufficient statistics release:** Adds symmetric Gaussian noise to $X^T X$ (scale $\sigma_G = C_x^2 \sqrt{2\log(1.25/\delta_2)}/\epsilon_2$) and to $X^T Y$ (scale $\sigma_h = C_x C_y \sqrt{2\log(1.25/\delta_3)}/\epsilon_3$).

5. **Private estimator computation:** Outputs
\[
\hat{\theta} = (\widetilde{G} + \lambda I)^{-1} \widetilde{h}
\]

All downstream computations are pure post-processing, incurring no additional privacy cost [2601.07545, 2405.15002].

## 3. Sensitivity Analysis and Noise Calibration

AdaSSP's privacy mechanisms use the Gaussian mechanism, requiring precise sensitivity estimates:
- Sensitivity of $G = X^T X$: $\Delta_F(G) \leq C_x^2$ (Frobenius norm).
- Sensitivity of $h = X^T Y$: $\Delta_2(h) \leq C_x C_y$.
- Sensitivity for the smallest eigenvalue: $\|X\|^2$ (from Weyl's inequality).

Noise scales are then derived as:
\[
\sigma = \Delta \cdot \sqrt{2\log(1.25/\delta)} / \epsilon
\]
applied separately for each release [2405.15002, 1803.02596].

## 4. Differential Privacy and Composition

AdaSSP's privacy guarantee follows from composition of Gaussian mechanisms:
- Each statistic release is $(\epsilon_i, \delta_i)$-DP.
- By advanced (or simple) composition, the aggregate procedure is $(\epsilon, \delta)$-DP with $\epsilon = \epsilon_1 + \epsilon_2 + \epsilon_3$, $\delta = \delta_1 + \delta_2 + \delta_3$ [2601.07545].
- All subsequent steps (ridge selection, inversion) are post-processing.

Empirical best practices apply analytic calibration to minimize noise for each mechanism [2601.07545].

## 5. Utility Analysis and Optimality

AdaSSP achieves rates that match data-dependent minimax lower bounds for excess risk:
- For empirical risk $R(X, Y, \hat{\theta}) = L(\hat{\theta}) - L(\theta^*)$,
\[
R(X, Y, \hat{\theta}) \leq O\left( \frac{Y_{\text{ASSP}} (C_y^2 + \|\theta^*\|_2^2)}{\lambda_{\min}(X^T X)} \right)
\]
for poorly-conditioned design, or
\[
O\left( Y_{\text{ASSP}} (C_y^2 + \|\theta^*\|_2^2) \right)
\]
otherwise, with
\[
Y_{\text{ASSP}} = O\left( \frac{ \sqrt{d\log(d^2/\delta)\log(1/\delta)} }{ \epsilon } \right) 
\]
[2601.07545].

Under standard Gaussian linear models, estimation error achieves asymptotic efficiency (matching Cramér–Rao up to $1/\epsilon^2$) [1803.02596]. AdaSSP adapts to the regime (Lipschitz or strongly convex) and recovers optimal rates in both.

## 6. Practical Considerations and Extensions

- The only tuning parameter is $\lambda$, which may be set via internal criteria to ensure invertibility or by cross-validation on the privatized statistics.
- Internal confidence ($\rho$) affects graceful tradeoff between utility and risk.
- Standardization of features ($\max_i \|x_i\| = 1$) is commonly performed. Empirical studies use settings such as $\epsilon \in \{0.1, 1\}$, $\delta = \min\{10^{-6}, 1/n^2\}$ [1803.02596].
- AdaSSP requires only norm bounds ($C_x, C_y$) and does not need user-side hyperparameter tuning.
- Approximate sufficient-statistics perturbation extends to logistic regression via Chebyshev approximation of the log-likelihood, where noisy $X^T X$ and $X^T Y$ suffice for quadratic approximations [2405.15002].

## 7. Comparative Evaluation and Empirical Performance

Extensive experiments on 36 benchmark UCI data sets demonstrate that AdaSSP uniformly improves over non-adaptive SSP and objective perturbation (ObjPert) at small and moderate privacy budgets, and matches or exceeds output perturbation sampling (OPS) and noisy SGD in nearly all cases. In Gaussian regime simulation, AdaSSP achieves near-1 relative efficiency at moderate $n$ ("privacy for free") [1803.02596].

AdaSSP's performance degrades gracefully with increasing $d$, decreasing $\epsilon$, or ill-conditioning. It remains the leading baseline among sufficient-statistics perturbation approaches unless the design is extremely well-conditioned or the residual error $\|Y - X\theta^*\|^2$ is negligible [2601.07545].

---

**References:**  
- "Near-Optimal Private Linear Regression via Iterative Hessian Mixing" [2601.07545]
- "Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domain" [1803.02596]
- "Private Regression via Data-Dependent Sufficient Statistic Perturbation" [2405.15002]

Source: https://www.emergentmind.com/topics/adaptive-sufficient-statistics-perturbation-adassp