---
title: Sketched Gaussian Mechanism (SGM)
url: https://www.emergentmind.com/topics/sketched-gaussian-mechanism-sgm
type: topic
---

# Sketched Gaussian Mechanism (SGM)

Searching arXiv for recent and foundational papers on “Sketched Gaussian Mechanism” and related usage.
Sketched Gaussian Mechanism (SGM) is an overloaded term in differential privacy and private machine learning. In one line of work, SGM denotes the **Sampled Gaussian Mechanism**, obtained by subsampling a dataset and then adding spherical Gaussian noise; this is the mechanism analyzed through Rényi Differential Privacy (RDP) in "Rényi Differential Privacy of the Sampled Gaussian Mechanism" [1908.10530]. In later work on private regression, federated learning, and continual observation, closely related terminology denotes mechanisms that first apply a **Gaussian sketch** or other linear sketch and then add Gaussian noise in the sketch space, such as the Gaussian Mixing Mechanism \(M(X)=\mathsf{S}X+\sigma\xi\) [2505.24603], the federated-learning mechanism \(\mathcal{SG}(\theta;R,\xi)=R\theta+\xi\) [2509.08195], and sketch-based continual-release mechanisms built from CountSketch and correlated Gaussian noise [2606.11760]. The common theme is the combination of linear randomization and Gaussian perturbation, but the privacy geometry, accounting methods, and utility trade-offs differ substantially across these usages.

## 1. Terminological scope and core variants

The terminology is not uniform across the literature. The 2019 RDP paper calls the object of study the **Sampled Gaussian Mechanism (SGM)**, and notes that in many machine-learning papers the same construction is also called *subsampled Gaussian mechanism*, *mini-batch Gaussian mechanism*, or colloquially a “sketched” Gaussian mechanism because the mechanism first selects a random “sketch” of the dataset and then applies a Gaussian mechanism to a function of that sketch [1908.10530]. By contrast, later regression and federated-learning papers use “sketched Gaussian mechanism” for mechanisms that apply a random Gaussian projection to vector or matrix data and add Gaussian noise in the lower-dimensional space [2505.24603, 2509.08195].

| Usage | Mechanism form | Primary setting |
|---|---|---|
| Sampled Gaussian Mechanism | subsample, then add \(\mathcal{N}(0,\sigma^2 I)\) | DP-SGD, privacy amplification |
| Gaussian Mixing / Gaussian sketching | \(\mathsf{S}X+\sigma\xi\) | private linear and logistic regression |
| Sketched Gaussian Mechanism in FL | \(R\theta+\xi\) | client-level private federated learning |
| Sketch + Gaussian continual release | sketch counters + Gaussian BTM noise | streaming, range counting, join estimation |

A central misconception is to treat these as identical mechanisms. They are not. The sampled mechanism derives privacy amplification from random **inclusion probability** \(q\); the regression and federated-learning versions derive privacy from random **projection geometry**, output dimension, or covariance structure; and the continual-observation construction derives privacy from a **correlated Gaussian process** implementing the binary tree mechanism [1908.10530, 2505.24603, 2509.08195, 2606.11760].

## 2. Sampled Gaussian Mechanism: definition and privacy model

For datasets \(S \in \mathcal{S}\), the sampled mechanism uses add/remove adjacency: \(S\) and \(S'\) are adjacent if \(S' = S \cup \{x\}\) or vice versa. A query \(f\) maps subsets of \(\mathcal{S}\) to \(\mathbb{R}^d\), with \(\ell_2\)-sensitivity \(1\),
\[
\|f(S)-f(S')\|_2 \le 1
\quad\text{for all adjacent } S,S'.
\]
Under Poisson subsampling with inclusion probability \(q \in (0,1]\), the mechanism outputs
\[
SG_{q,\sigma}(S)
=
f\big(\{x \in S : x \text{ is sampled with prob } q\}\big)
+
\mathcal{N}(0,\sigma^2 I_d).
\]
This is a composition of random subsampling and a standard Gaussian mechanism [1908.10530].

RDP is the natural accounting framework for this mechanism. For \(\alpha>1\), the Rényi divergence is
\[
D_\alpha(P\|Q)=\frac{1}{\alpha-1}\ln \int q(x)\Big(\frac{p(x)}{q(x)}\Big)^\alpha dx,
\]
and a mechanism satisfies \((\alpha,\varepsilon)\)-RDP if the divergence between outputs on adjacent datasets is at most \(\varepsilon\), symmetrically. The attraction of RDP for SGM is threefold: composition is additive at fixed order \(\alpha\), it is equivalent to controlling the log moment generating function of privacy loss, and it converts to \((\varepsilon,\delta)\)-DP by optimizing
\[
\varepsilon(\delta)=\min_{\alpha>1}\left\{\varepsilon(\alpha)+\frac{\ln(1/\delta)}{\alpha-1}\right\}.
\]
This is exactly the accounting structure used in DP-SGD and in moments-accountant analyses recast in RDP form [1908.10530].

## 3. RDP characterization and quadratic privacy amplification

The main technical reduction for the sampled mechanism is to a one-dimensional Gaussian mixture. Let
\[
\mu_0=\mathcal{N}(0,\sigma^2),\qquad
\mu_1=\mathcal{N}(1,\sigma^2),\qquad
\mu=(1-q)\mu_0+q\mu_1.
\]
If \(f\) has \(\ell_2\)-sensitivity \(1\), then for adjacent \(S,S'\),
\[
D_\alpha(M(S)\|M(S')) \le D_\alpha(\mu_0\|\mu),\qquad
D_\alpha(M(S')\|M(S)) \le D_\alpha(\mu\|\mu_0).
\]
Defining
\[
A_\alpha = \mathbf{E}_{z\sim\mu_0}\Big[\big(\mu(z)/\mu_0(z)\big)^\alpha\Big],\qquad
B_\alpha = \mathbf{E}_{z\sim\mu}\Big[\big(\mu_0(z)/\mu(z)\big)^\alpha\Big],
\]
the paper proves the dominance relation \(A_\alpha \ge B_\alpha\) for all \(\alpha \ge 1\), so bounding \(A_\alpha\) suffices [1908.10530].

Under the conditions \(q\le 1/5\), \(\sigma\ge 4\), and
\[
1<\alpha\le \frac12 \sigma^2 L - 2\ln\sigma,
\]
\[
\alpha\le
\frac{\frac12 \sigma^2 L^2 - \ln 5 - 2\ln\sigma}
     {L + \ln(q\alpha) + \frac{1}{2\sigma^2}},
\qquad
L=\ln\!\left(1+\frac{1}{q(\alpha-1)}\right),
\]
SGM satisfies the closed-form RDP bound
\[
\varepsilon(\alpha)\le \frac{2q^2\alpha}{\sigma^2}.
\]
The paper describes this as a nearly tight closed-form bound, with correct \(O(q^2\alpha/\sigma^2)\) dependence and numerical comparisons showing only a small constant-factor gap in the regime where the theorem applies [1908.10530].

The principal significance is the **quadratic privacy amplification** in \(q\). For the standard Gaussian mechanism with sensitivity \(1\),
\[
\varepsilon_{\mathrm{Gauss}}(\alpha)=\frac{\alpha}{2\sigma^2},
\]
whereas for SGM,
\[
\varepsilon_{\mathrm{SGM}}(\alpha)\lesssim \frac{2q^2\alpha}{\sigma^2}.
\]
Hence, for small sampling rates,
\[
\varepsilon_{\mathrm{SGM}}(\alpha)=\Theta(q^2\alpha/\sigma^2),
\]
rather than the linear-in-\(q\) behavior suggested by a naive intuition. The paper also gives an exact, numerically stable procedure for arbitrary \(\alpha>1\): for integer \(\alpha\), \(A_\alpha\) is a finite sum
\[
A_\alpha
=
\sum_{k=0}^{\alpha}
\binom{\alpha}{k}(1-q)^{\alpha-k}q^k
\exp\!\left(\frac{k^2-k}{2\sigma^2}\right),
\]
and for fractional \(\alpha\), \(A_\alpha\) is computed through absolutely convergent series with generalized binomial coefficients and \(\operatorname{erfc}\) terms, typically accumulated in log-space. This numerical accountant is the basis of practical RDP accounting for DP-SGD, including implementations such as TensorFlow Privacy [1908.10530].

## 4. Gaussian sketching and Gaussian mixing for regression

A distinct SGM lineage studies Gaussian sketching as the privacy mechanism itself. In "The Gaussian Mixing Mechanism: Renyi Differential Privacy via Gaussian Sketches" [2505.24603], the dataset is a matrix \(X\in\mathbb{R}^{n\times d}\) with row bound \(\|x_i\|\le C_X\), zero-out neighboring relation, and a lower bound \(\overline{\lambda}_{\min}\le \lambda_{\min}(X^\top X)\). The mechanism is
\[
M(X)=\mathsf{S}X+\sigma\xi,
\qquad
\mathsf{S}\sim\mathcal{N}(0,\mathbf{I}_{k\times n}),
\quad
\xi\sim\mathcal{N}(0,\mathbf{I}_{k\times d}),
\]
and may also be applied jointly to \((X_1,X_2)\) and post-processed into
\[
K(M(X_1,X_2))
=
(\mathsf{S}X_1+\sigma\xi_1)^\top(\mathsf{S}X_2+\sigma\xi_2),
\]
which yields private sufficient statistics such as \(X^\top X\) and \(X^\top y\) [2505.24603].

The RDP analysis depends on
\[
\gamma=\frac{\overline{\lambda}_{\min}+\sigma^2}{C_X^2}.
\]
For \(\gamma>1\), GaussMix satisfies \((\alpha,\varphi(\alpha;k,\gamma))\)-RDP for all \(\alpha\in(1,\gamma)\), where
\[
\varphi(\alpha;k,\zeta)
=
\frac{k\alpha}{2(\alpha-1)}\log\!\Big(1-\frac{1}{\zeta}\Big)
-
\frac{k}{2(\alpha-1)}\log\!\Big(1-\frac{\alpha}{\zeta}\Big),
\qquad \zeta>1.
\]
This explicit RDP curve comes from viewing \(M(X)^\top\) as having i.i.d. Gaussian columns with covariance \(\Sigma=X^\top X+\sigma^2 I_d\), comparing neighboring covariances \(\Sigma\) and \(\Sigma-x_1x_1^\top\), and reducing the divergence to a determinant formula. Privacy improves as \(\lambda_{\min}(X^\top X)\) and \(\sigma^2\) increase, both through \(\gamma\) [2505.24603].

The same paper shows that if \(\gamma>5/2\), GaussMix satisfies \((\rho,w)\)-tCDP with
\[
\rho=\frac{k}{2\gamma^2},\qquad w=\frac{2\gamma}{5},
\]
and compares favorably with Sheffet’s earlier \((\varepsilon,\delta)\)-DP analysis by eliminating an extra linear term in \(\log(1/\delta)\). It then builds instance-dependent regression algorithms: **ModifiedGaussianMix** privately estimates \(\lambda_{\min}(X^\top X)\), adjusts the noise level accordingly, and **LinearMixing** solves OLS on the sketch; a quadratic-surrogate reduction yields a corresponding **LogisticMix** procedure for logistic regression [2505.24603].

The utility theory is equally instance-sensitive. For OLS, Theorem 4.3 bounds excess empirical loss by a term scaling with
\[
(1+\chi)^2
\frac{\sqrt{k\log(1/\delta)}(C_X^2+C_Y^2)}{\varepsilon}
(1+\|\theta^*\|^2),
\]
up to universal constants and probability terms, while the regime \(k\sim d\log(1/\varrho)\) matches known optimal AdaSSP bounds when \(\lambda_{\min}(X^\top X)=0\). Empirically, the paper reports lower test MSE than Sheffet’s earlier SGM-based OLS and competitive runtime, and for logistic regression on private embeddings it reports higher classification accuracy and faster execution than objective perturbation [2505.24603].

## 5. Sketched Gaussian Mechanism in federated learning

In federated learning, the Sketched Gaussian Mechanism is defined directly on a vector statistic:
\[
\mathcal{SG}(\theta;R,\xi)=R\theta+\xi,
\]
where \(R\in\mathbb{R}^{b\times d}\) is a Gaussian sketching matrix with entries \(R_{ij}\sim\mathcal{N}(0,1/\sqrt{b})\) i.i.d., and \(\xi\sim\mathcal{N}(0,\sigma_g^2 I_b)\). Here \(b\) is the sketching dimension, typically much smaller than \(d\). In minibatch DP-SGD, clipped per-example gradients are sketched, perturbed, and aggregated in sketch space; at batch level, the mechanism acts on the clipped sum \(\gamma_t(D)\) with covariance determined by \(\|\gamma_t(D)\|_2^2/b + m\sigma_g^2\) [2509.08195].

The privacy analysis is joint rather than compositional. For neighboring datasets, the outputs are zero-mean Gaussians with different variances, and the divergence is expressed as
\[
D_\alpha\bigl(\mathcal{SG}(\gamma_t(D))\|\mathcal{SG}(\gamma_t(D'))\bigr)=b\,f_\alpha(x),
\]
with the variance ratio controlled through a **ratio sensitivity**
\[
\operatorname{rsens}_c(\theta)
=
\sup_{D\sim D'}
\sqrt{\frac{\|\theta(D')\|_2^2+c^2}{\|\theta(D)\|_2^2+c^2}}.
\]
For clipped sums with \(\|\hat g_t(x_i)\|_2\le \tau\), the one-step mechanism is shown to satisfy
\[
\epsilon_\alpha
=
\frac{\alpha^2\tau^4}{(\alpha-1)\,b\,\sigma_g^4},
\]
for suitable parameter regimes. After subsampling and composition, Theorem 3.2 gives an \((\varepsilon_p,\delta_p)\)-DP guarantee when
\[
\sigma_g^2
\ge
\frac{c_4 q\tau^2\sqrt{T}\log(2qT/\delta_p)}
{\sqrt{b}\,\varepsilon_p}.
\]
Equivalently, for fixed noise magnitude, the privacy level is proportional to \(1/\sqrt{b}\), which is the paper’s defining privacy amplification result [2509.08195].

This analysis differs sharply from the classical Gaussian mechanism. For plain GM applied directly in \(\mathbb{R}^d\), the RDP bound does not depend on \(d\); for SGM, the privacy guarantee depends on the sketching dimension \(b\), and a joint analysis of sketching and noise yields stronger privacy than treating sketching and Gaussian perturbation as isolated mechanisms. The paper is explicit that the theory is specific to the **sketch-then-noise** order; it does not claim equivalence under reordering [2509.08195].

The federated-learning instantiation, **Fed-SGM**, combines client sampling, local training, clipping, sketching, sketch-space Gaussian noise, desketching by \(R_t^\top\), and either server-side GD or AMSGrad. Under bounded gradients, sub-Gaussian stochastic noise, smoothness, and an absolute intrinsic dimension assumption
\[
\frac{\sum_{i=1}^d |\lambda_i|}{\max_i \lambda_i}\le \mathcal{I},
\]
the convergence bounds have at most logarithmic dependence on \(d\) and linear dependence on \(\mathcal{I}\). Experiments on EMNIST ByClass with ResNet101 and SST-2 with BERT-Base report that Fed-SGM is at least competitive with unsketched DP-FedAvg at the same privacy level and can outperform it, while also reducing client communication from \(O(d)\) to \(O(b)\) [2509.08195].

## 6. Continual observation, CountSketch, and correlated Gaussian release

A further extension places sketched Gaussian mechanisms in the continual-observation model. "A Fast Gaussian Mechanism under Continual Observation, with Applications" [2606.11760] studies updates \(x^{(1)},x^{(2)},\dots\) to a \(k\)-dimensional vector over time, with neighboring datasets differing in at most one update and each update bounded in norm. The classical Gaussian binary tree mechanism can release every partial sum \(A^{(t)}\) with polylogarithmic noise, but standard implementations cost \(O(\log T)\) time per query. The paper gives a data structure that samples any desired noise entry in **constant time** while reproducing exactly the distribution of the Gaussian binary tree mechanism, using Brownian bridges [2606.11760].

The binary-tree view writes the released sequence as
\[
\tilde A = L(Rx+N),
\qquad
N\sim\mathcal{N}(0,\sigma^2 I_{2T-1}),
\]
with strategy matrix sensitivity
\[
\Delta_2(R)=\frac12\sqrt{\log_2 T+2}.
\]
If
\[
\sigma^2=\frac{\log_2 T+2}{8\rho},
\]
then the release is \(\rho\)-zCDP, and each time-\(t\) answer has variance
\[
\operatorname{Var}[\tilde A^{(t)}]
=
(\log_2 T+1)\sigma^2
<
\frac{(\log_2 T+2)^2}{8\rho}.
\]
The new contribution is algorithmic: instead of materializing all node noises, the mechanism stores partial sums on the current root-to-leaf path and uses the Brownian bridge conditional law
\[
S(m)\mid S(a),S(c)
\sim
\mathcal{N}\!\left(
x_a+\frac{m-a}{c-a}(x_c-x_a),
\frac{(m-a)(c-m)}{c-a}\sigma^2
\right)
\]
to sample only the missing shared prefix needed for the next query [2606.11760].

This machinery becomes a sketched Gaussian mechanism when combined with **Private CountSketch**. In the static case, CountSketch is a linear map \(CS:\mathbb{R}^d\to\mathbb{R}^{t\times b}\), and adding independent Gaussian noise to each sketch cell yields \(\rho\)-zCDP with
\[
\rho=\frac{t}{2\sigma^2},
\]
because the flattened sketch has \(\ell_2\)-sensitivity \(\sqrt{t}\). Under continual observation, each sketch cell is equipped with its own FastGaMe process, so the sketch counters evolve with exactly the correlated Gaussian noise of the binary tree mechanism. The paper uses this construction for dynamic orthogonal range counting and join-size estimation, obtaining \(\rho\)-zCDP, polylogarithmic error, and constant-time noise generation per touched sketch cell [2606.11760].

## 7. Assumptions, limitations, and conceptual synthesis

Across these literatures, SGM is powerful but assumption-sensitive. The sampled mechanism of [1908.10530] is proved primarily for **Poisson subsampling**, \(\ell_2\)-sensitivity-bounded queries, and Gaussian noise; its clean closed-form theorem requires \(q\le 1/5\), \(\sigma\ge 4\), and nontrivial conditions on \(\alpha\), and outside that regime one should use exact numerical accounting. The Gaussian-mixing mechanism of [2505.24603] relies on a row norm bound, zero-out adjacency, and for the strongest statements a lower bound on \(\lambda_{\min}(X^\top X)\). The federated-learning SGM of [2509.08195] assumes dense Gaussian sketches, independent randomness across rounds, clipping, and specific covariance-based RDP calculations; the paper states that extending the theory to CountSketch, sparse JL, or structured transforms is open. The continual-observation framework of [2606.11760] relies on binary-tree Gaussian correlations and, in the applications, CountSketch-specific sensitivity calculations.

A second misconception is that every SGM behaves like a standard Gaussian mechanism with a modified variance. This is only partially true. For the sampled mechanism, the leading privacy term behaves like \(O(q^2\alpha/\sigma^2)\), reflecting privacy amplification by subsampling [1908.10530]. For GaussMix, the relevant quantity is the spectrum of \(X^\top X+\sigma^2 I_d\), summarized by \(\gamma\) [2505.24603]. For federated-learning SGM, privacy depends on the sketching dimension through a \(1/\sqrt{b}\) law after composition [2509.08195]. For continual observation, the mechanism is not i.i.d. Gaussian noise per time step but a correlated Gaussian process engineered for polylogarithmic composition [2606.11760].

A plausible implication is that “SGM” is best read as a contextual label for a **family** of mechanisms that combine random sketching or sampling with Gaussian perturbation, rather than as a single canonical construction. What unifies these variants is the use of RDP, tCDP, or zCDP to exploit structure that ordinary \((\varepsilon,\delta)\)-DP analyses often obscure: mixture structure in subsampling, covariance structure in Gaussian sketching, dimension reduction in federated learning, and correlation structure in continual release.

Source: https://www.emergentmind.com/topics/sketched-gaussian-mechanism-sgm