Sketched Gaussian Mechanism (SGM)

• SGM is a family of mechanisms that combine random subsampling or sketching with Gaussian perturbation to ensure differential privacy in various machine learning applications.
• Its variants achieve privacy amplification through different routes—random inclusion probability, projection geometry, or noise correlation—tailoring the approach for DP-SGD, private regression, and federated learning.
• Rigorous RDP and tCDP analyses provide nearly tight privacy bounds with quadratic amplification effects and dimension-dependent trade-offs, enabling effective continual observation and federated model training.

Searching arXiv for recent and foundational papers on “Sketched Gaussian Mechanism” and related usage. Sketched Gaussian Mechanism (SGM) is an overloaded term in differential privacy and private machine learning. In one line of work, SGM denotes the Sampled Gaussian Mechanism, obtained by subsampling a dataset and then adding spherical Gaussian noise; this is the mechanism analyzed through Rényi Differential Privacy (RDP) in "Rényi Differential Privacy of the Sampled Gaussian Mechanism" (Mironov et al., 2019). In later work on private regression, federated learning, and continual observation, closely related terminology denotes mechanisms that first apply a Gaussian sketch or other linear sketch and then add Gaussian noise in the sketch space, such as the Gaussian Mixing Mechanism $M(X)=\mathsf{S}X+\sigma\xi$ (Lev et al., 30 May 2025), the federated-learning mechanism $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$ (Li et al., 9 Sep 2025), and sketch-based continual-release mechanisms built from CountSketch and correlated Gaussian noise (Pagh et al., 10 Jun 2026). The common theme is the combination of linear randomization and Gaussian perturbation, but the privacy geometry, accounting methods, and utility trade-offs differ substantially across these usages.

1. Terminological scope and core variants

The terminology is not uniform across the literature. The 2019 RDP paper calls the object of study the Sampled Gaussian Mechanism (SGM), and notes that in many machine-learning papers the same construction is also called subsampled Gaussian mechanism, mini-batch Gaussian mechanism, or colloquially a “sketched” Gaussian mechanism because the mechanism first selects a random “sketch” of the dataset and then applies a Gaussian mechanism to a function of that sketch (Mironov et al., 2019). By contrast, later regression and federated-learning papers use “sketched Gaussian mechanism” for mechanisms that apply a random Gaussian projection to vector or matrix data and add Gaussian noise in the lower-dimensional space (Lev et al., 30 May 2025, Li et al., 9 Sep 2025).

Usage	Mechanism form	Primary setting
Sampled Gaussian Mechanism	subsample, then add $\mathcal{N}(0,\sigma^2 I)$	DP-SGD, privacy amplification
Gaussian Mixing / Gaussian sketching	$\mathsf{S}X+\sigma\xi$	private linear and logistic regression
Sketched Gaussian Mechanism in FL	$R\theta+\xi$	client-level private federated learning
Sketch + Gaussian continual release	sketch counters + Gaussian BTM noise	streaming, range counting, join estimation

A central misconception is to treat these as identical mechanisms. They are not. The sampled mechanism derives privacy amplification from random inclusion probability $q$; the regression and federated-learning versions derive privacy from random projection geometry, output dimension, or covariance structure; and the continual-observation construction derives privacy from a correlated Gaussian process implementing the binary tree mechanism (Mironov et al., 2019, Lev et al., 30 May 2025, Li et al., 9 Sep 2025, Pagh et al., 10 Jun 2026).

2. Sampled Gaussian Mechanism: definition and privacy model

For datasets $S \in \mathcal{S}$, the sampled mechanism uses add/remove adjacency: $S$ and $S'$ are adjacent if $S' = S \cup \{x\}$ or vice versa. A query $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$0 maps subsets of $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$1 to $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$2, with $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$3-sensitivity $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$4,

$\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$5

Under Poisson subsampling with inclusion probability $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$6, the mechanism outputs

$\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$7

This is a composition of random subsampling and a standard Gaussian mechanism (Mironov et al., 2019).

RDP is the natural accounting framework for this mechanism. For $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$8, the Rényi divergence is

$\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$9

and a mechanism satisfies $\mathcal{N}(0,\sigma^2 I)$0-RDP if the divergence between outputs on adjacent datasets is at most $\mathcal{N}(0,\sigma^2 I)$1, symmetrically. The attraction of RDP for SGM is threefold: composition is additive at fixed order $\mathcal{N}(0,\sigma^2 I)$2, it is equivalent to controlling the log moment generating function of privacy loss, and it converts to $\mathcal{N}(0,\sigma^2 I)$3-DP by optimizing

$\mathcal{N}(0,\sigma^2 I)$4

This is exactly the accounting structure used in DP-SGD and in moments-accountant analyses recast in RDP form (Mironov et al., 2019).

3. RDP characterization and quadratic privacy amplification

The main technical reduction for the sampled mechanism is to a one-dimensional Gaussian mixture. Let

$\mathcal{N}(0,\sigma^2 I)$5

If $\mathcal{N}(0,\sigma^2 I)$6 has $\mathcal{N}(0,\sigma^2 I)$7-sensitivity $\mathcal{N}(0,\sigma^2 I)$8, then for adjacent $\mathcal{N}(0,\sigma^2 I)$9,

$\mathsf{S}X+\sigma\xi$0

Defining

$\mathsf{S}X+\sigma\xi$1

the paper proves the dominance relation $\mathsf{S}X+\sigma\xi$2 for all $\mathsf{S}X+\sigma\xi$3, so bounding $\mathsf{S}X+\sigma\xi$4 suffices (Mironov et al., 2019).

Under the conditions $\mathsf{S}X+\sigma\xi$5, $\mathsf{S}X+\sigma\xi$6, and

$\mathsf{S}X+\sigma\xi$7

$\mathsf{S}X+\sigma\xi$8

SGM satisfies the closed-form RDP bound

$\mathsf{S}X+\sigma\xi$9

The paper describes this as a nearly tight closed-form bound, with correct $R\theta+\xi$0 dependence and numerical comparisons showing only a small constant-factor gap in the regime where the theorem applies (Mironov et al., 2019).

The principal significance is the quadratic privacy amplification in $R\theta+\xi$1. For the standard Gaussian mechanism with sensitivity $R\theta+\xi$2,

$R\theta+\xi$3

whereas for SGM,

$R\theta+\xi$4

Hence, for small sampling rates,

$R\theta+\xi$5

rather than the linear-in-$R\theta+\xi$6 behavior suggested by a naive intuition. The paper also gives an exact, numerically stable procedure for arbitrary $R\theta+\xi$7: for integer $R\theta+\xi$8, $R\theta+\xi$9 is a finite sum

$q$0

and for fractional $q$1, $q$2 is computed through absolutely convergent series with generalized binomial coefficients and $q$3 terms, typically accumulated in log-space. This numerical accountant is the basis of practical RDP accounting for DP-SGD, including implementations such as TensorFlow Privacy (Mironov et al., 2019).

4. Gaussian sketching and Gaussian mixing for regression

A distinct SGM lineage studies Gaussian sketching as the privacy mechanism itself. In "The Gaussian Mixing Mechanism: Renyi Differential Privacy via Gaussian Sketches" (Lev et al., 30 May 2025), the dataset is a matrix $q$4 with row bound $q$5, zero-out neighboring relation, and a lower bound $q$6. The mechanism is

$q$7

and may also be applied jointly to $q$8 and post-processed into

$q$9

which yields private sufficient statistics such as $S \in \mathcal{S}$0 and $S \in \mathcal{S}$1 (Lev et al., 30 May 2025).

The RDP analysis depends on

$S \in \mathcal{S}$2

For $S \in \mathcal{S}$3, GaussMix satisfies $S \in \mathcal{S}$4-RDP for all $S \in \mathcal{S}$5, where

$S \in \mathcal{S}$6

This explicit RDP curve comes from viewing $S \in \mathcal{S}$7 as having i.i.d. Gaussian columns with covariance $S \in \mathcal{S}$8, comparing neighboring covariances $S \in \mathcal{S}$9 and $S$0, and reducing the divergence to a determinant formula. Privacy improves as $S$1 and $S$2 increase, both through $S$3 (Lev et al., 30 May 2025).

The same paper shows that if $S$4, GaussMix satisfies $S$5-tCDP with

$S$6

and compares favorably with Sheffet’s earlier $S$7-DP analysis by eliminating an extra linear term in $S$8. It then builds instance-dependent regression algorithms: ModifiedGaussianMix privately estimates $S$9, adjusts the noise level accordingly, and LinearMixing solves OLS on the sketch; a quadratic-surrogate reduction yields a corresponding LogisticMix procedure for logistic regression (Lev et al., 30 May 2025).

The utility theory is equally instance-sensitive. For OLS, Theorem 4.3 bounds excess empirical loss by a term scaling with

$S'$0

up to universal constants and probability terms, while the regime $S'$1 matches known optimal AdaSSP bounds when $S'$2. Empirically, the paper reports lower test MSE than Sheffet’s earlier SGM-based OLS and competitive runtime, and for logistic regression on private embeddings it reports higher classification accuracy and faster execution than objective perturbation (Lev et al., 30 May 2025).

5. Sketched Gaussian Mechanism in federated learning

In federated learning, the Sketched Gaussian Mechanism is defined directly on a vector statistic: $S'$3 where $S'$4 is a Gaussian sketching matrix with entries $S'$5 i.i.d., and $S'$6. Here $S'$7 is the sketching dimension, typically much smaller than $S'$8. In minibatch DP-SGD, clipped per-example gradients are sketched, perturbed, and aggregated in sketch space; at batch level, the mechanism acts on the clipped sum $S'$9 with covariance determined by $S' = S \cup \{x\}$0 (Li et al., 9 Sep 2025).

The privacy analysis is joint rather than compositional. For neighboring datasets, the outputs are zero-mean Gaussians with different variances, and the divergence is expressed as

$S' = S \cup \{x\}$1

with the variance ratio controlled through a ratio sensitivity

$S' = S \cup \{x\}$2

For clipped sums with $S' = S \cup \{x\}$3, the one-step mechanism is shown to satisfy

$S' = S \cup \{x\}$4

for suitable parameter regimes. After subsampling and composition, Theorem 3.2 gives an $S' = S \cup \{x\}$5-DP guarantee when

$S' = S \cup \{x\}$6

Equivalently, for fixed noise magnitude, the privacy level is proportional to $S' = S \cup \{x\}$7, which is the paper’s defining privacy amplification result (Li et al., 9 Sep 2025).

This analysis differs sharply from the classical Gaussian mechanism. For plain GM applied directly in $S' = S \cup \{x\}$8, the RDP bound does not depend on $S' = S \cup \{x\}$9; for SGM, the privacy guarantee depends on the sketching dimension $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$00, and a joint analysis of sketching and noise yields stronger privacy than treating sketching and Gaussian perturbation as isolated mechanisms. The paper is explicit that the theory is specific to the sketch-then-noise order; it does not claim equivalence under reordering (Li et al., 9 Sep 2025).

The federated-learning instantiation, Fed-SGM, combines client sampling, local training, clipping, sketching, sketch-space Gaussian noise, desketching by $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$01, and either server-side GD or AMSGrad. Under bounded gradients, sub-Gaussian stochastic noise, smoothness, and an absolute intrinsic dimension assumption

$\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$02

the convergence bounds have at most logarithmic dependence on $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$03 and linear dependence on $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$04. Experiments on EMNIST ByClass with ResNet101 and SST-2 with BERT-Base report that Fed-SGM is at least competitive with unsketched DP-FedAvg at the same privacy level and can outperform it, while also reducing client communication from $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$05 to $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$06 (Li et al., 9 Sep 2025).

6. Continual observation, CountSketch, and correlated Gaussian release

A further extension places sketched Gaussian mechanisms in the continual-observation model. "A Fast Gaussian Mechanism under Continual Observation, with Applications" (Pagh et al., 10 Jun 2026) studies updates $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$07 to a $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$08-dimensional vector over time, with neighboring datasets differing in at most one update and each update bounded in norm. The classical Gaussian binary tree mechanism can release every partial sum $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$09 with polylogarithmic noise, but standard implementations cost $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$10 time per query. The paper gives a data structure that samples any desired noise entry in constant time while reproducing exactly the distribution of the Gaussian binary tree mechanism, using Brownian bridges (Pagh et al., 10 Jun 2026).

The binary-tree view writes the released sequence as

$\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$11

with strategy matrix sensitivity

$\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$12

If

$\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$13

then the release is $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$14-zCDP, and each time-$\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$15 answer has variance

$\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$16

The new contribution is algorithmic: instead of materializing all node noises, the mechanism stores partial sums on the current root-to-leaf path and uses the Brownian bridge conditional law

$\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$17

to sample only the missing shared prefix needed for the next query (Pagh et al., 10 Jun 2026).

This machinery becomes a sketched Gaussian mechanism when combined with Private CountSketch. In the static case, CountSketch is a linear map $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$18, and adding independent Gaussian noise to each sketch cell yields $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$19-zCDP with

$\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$20

because the flattened sketch has $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$21-sensitivity $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$22. Under continual observation, each sketch cell is equipped with its own FastGaMe process, so the sketch counters evolve with exactly the correlated Gaussian noise of the binary tree mechanism. The paper uses this construction for dynamic orthogonal range counting and join-size estimation, obtaining $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$23-zCDP, polylogarithmic error, and constant-time noise generation per touched sketch cell (Pagh et al., 10 Jun 2026).

7. Assumptions, limitations, and conceptual synthesis

Across these literatures, SGM is powerful but assumption-sensitive. The sampled mechanism of (Mironov et al., 2019) is proved primarily for Poisson subsampling, $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$24-sensitivity-bounded queries, and Gaussian noise; its clean closed-form theorem requires $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$25, $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$26, and nontrivial conditions on $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$27, and outside that regime one should use exact numerical accounting. The Gaussian-mixing mechanism of (Lev et al., 30 May 2025) relies on a row norm bound, zero-out adjacency, and for the strongest statements a lower bound on $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$28. The federated-learning SGM of (Li et al., 9 Sep 2025) assumes dense Gaussian sketches, independent randomness across rounds, clipping, and specific covariance-based RDP calculations; the paper states that extending the theory to CountSketch, sparse JL, or structured transforms is open. The continual-observation framework of (Pagh et al., 10 Jun 2026) relies on binary-tree Gaussian correlations and, in the applications, CountSketch-specific sensitivity calculations.

A second misconception is that every SGM behaves like a standard Gaussian mechanism with a modified variance. This is only partially true. For the sampled mechanism, the leading privacy term behaves like $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$29, reflecting privacy amplification by subsampling (Mironov et al., 2019). For GaussMix, the relevant quantity is the spectrum of $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$30, summarized by $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$31 (Lev et al., 30 May 2025). For federated-learning SGM, privacy depends on the sketching dimension through a $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$32 law after composition (Li et al., 9 Sep 2025). For continual observation, the mechanism is not i.i.d. Gaussian noise per time step but a correlated Gaussian process engineered for polylogarithmic composition (Pagh et al., 10 Jun 2026).

A plausible implication is that “SGM” is best read as a contextual label for a family of mechanisms that combine random sketching or sampling with Gaussian perturbation, rather than as a single canonical construction. What unifies these variants is the use of RDP, tCDP, or zCDP to exploit structure that ordinary $\mathcal{SG}(\theta;R,\xi)=R\theta+\xi$33-DP analyses often obscure: mixture structure in subsampling, covariance structure in Gaussian sketching, dimension reduction in federated learning, and correlation structure in continual release.