---
title: Rényi Differential Privacy (RDP)
url: https://www.emergentmind.com/topics/renyi-differential-privacy-rdp
type: topic
---

# Rényi Differential Privacy (RDP)

Rényi Differential Privacy (RDP) is a relaxation of differential privacy formalized by Mironov (2017) that parameterizes the privacy loss of randomized algorithms using Rényi divergence, which is a moment-based generalization of the classical max-divergence used in differential privacy. RDP enables tight and modular analysis of privacy composition—particularly for mechanisms employing subsampling, adaptive composition, or moments accountants—and is now the analytic foundation for most state-of-the-art private machine learning pipelines.

## 1. Formal Definition and Divergence

Let $P$ and $Q$ be two probability measures defined on the same measurable space. For any order $\alpha>1$, the Rényi divergence of order $\alpha$ is

\[
D_\alpha(P\|Q) = \frac{1}{\alpha-1} \log \mathbb{E}_{x\sim Q}\left[\left(\frac{P(x)}{Q(x)}\right)^\alpha\right].
\]

A randomized mechanism $M$ operating on databases satisfies $(\alpha,\varepsilon)$-Rényi Differential Privacy if for every pair of adjacent datasets $D,D'$ (differing in one element):

\[
D_\alpha\bigl(M(D)\|\;M(D')\bigr)\;\le\;\varepsilon.
\]

As $\alpha\to1$, $D_\alpha(P\|Q)$ approaches the Kullback–Leibler divergence; as $\alpha\to\infty$, $D_\alpha(P\|Q)$ approaches the max-divergence, recovering pure $\varepsilon$-differential privacy [1702.07476].

## 2. Composition, Conversion, and Accounting

### Sequential and Adaptive Composition

If mechanisms $M_1,\ldots,M_k$ each satisfy $(\alpha,\varepsilon_i)$-RDP (possibly with different $\varepsilon_i$ but the same $\alpha$), then their k-fold (possibly adaptive) composition is $(\alpha, \sum_{i=1}^k \varepsilon_i)$-RDP [1702.07476], [1808.00087]. This enables linear, order-preserving tracking of privacy loss under repeated or adaptive use of RDP mechanisms.

### Conversion to ($\varepsilon$, $\delta$)-Differential Privacy

Any mechanism guaranteeing $(\alpha,\varepsilon)$-RDP also satisfies $(\varepsilon+\frac{\log(1/\delta)}{\alpha-1},\ \delta)$–differential privacy for any $0 < \delta < 1$:

\[
\varepsilon' = \varepsilon + \frac{\log(1/\delta)}{\alpha-1}
\]
[1702.07476], [1808.00087], [2103.01379].

The parameter $\alpha$ is typically tuned to optimize $\varepsilon'$ for a fixed target $\delta$.

### Analytical Moments Accountant

The cumulant generating function (CGF) of the privacy loss for a mechanism $M$ is

\[
K_M(\lambda) = \log \mathbb{E}_{\theta\sim M(X')}\left[\left(\frac{M(X)(\theta)}{M(X')(\theta)}\right)^{\lambda+1}\right]
\]

with $\varepsilon(\alpha) = K_M(\alpha-1)/(\alpha-1)$. The moments accountant accumulates CGFs additively for composed mechanisms:

\[
K_{\textrm{tot}}(\lambda) = \sum_{i=1}^k K_{M_i}(\lambda)
\]

and supports tight ($\varepsilon$, $\delta$)-DP conversion via univariate convex optimization [1808.00087].

## 3. Amplification by Subsampling and Shuffling

Subsampling (selecting a random subset before applying a mechanism) and shuffling (randomly permuting outputs in the distributed local setting) both amplify privacy.

### Subsampling

For Poisson sampling at rate $\gamma$, then applying an $(\alpha,\varepsilon(\alpha))$-RDP mechanism $M$ to the sample, the composed mechanism is $(\alpha, \varepsilon'(\alpha))$-RDP, where

\[
\varepsilon'(\alpha) \le \frac{1}{\alpha - 1} \log\left(1 + \sum_{j=2}^\alpha \binom{\alpha}{j}\gamma^j c_j\right)
\]
and $c_j$ are derived from the moments of $M$ [1808.00087]. In the Gaussian case with sampling probability $q$ and noise $\sigma$, a tight closed form is $\varepsilon(\alpha) \le 2q^2\alpha/\sigma^2$ for valid regimes [1908.10530].

### Shuffling

Shuffling can quadratically amplify privacy in the number of records under local randomization. If $n$ users each apply an $\varepsilon_0$-LDP mechanism, then shuffling yields

\[
\varepsilon_{\textrm{shuf}}(\alpha) \le \frac{1}{\alpha-1}\log\left(1+\sum_{j=2}^{\alpha} \binom{\alpha}{j} \frac{(e^{\varepsilon_0} - 1)^j}{n^{j-1}}\right)
\]
[1907.05156], [2105.05180], [2401.04306]. For large $n$ and small $\varepsilon_0$, this is $O(\varepsilon_0^2\alpha/n)$.

Shuffle model analyses now reach closed-form expressions and asymptotically optimal bounds [2401.04306].

## 4. Mechanism-Specific RDP Guarantees

### Gaussian Mechanism

If $f$ has $\ell_2$-sensitivity $\Delta$, releasing $f(D) + \mathcal{N}(0, \sigma^2 I)$ gives

\[
D_\alpha(\mathcal{N}(0, \sigma^2)\|\mathcal{N}(\Delta, \sigma^2)) = \frac{\alpha \Delta^2}{2\sigma^2} 
\]
Therefore, the mechanism is $(\alpha, \alpha\Delta^2/(2\sigma^2))$-RDP [1702.07476], [1908.10530].

### Laplace Mechanism

If $f$ has $\ell_1$-sensitivity $\Delta$, releasing $f(D) + \mathrm{Laplace}(\lambda)$ yields

\[
D_\alpha(\mathrm{Lap}(0, \lambda)\|\mathrm{Lap}(\Delta, \lambda)) = \frac{1}{\alpha-1} \log\left(\frac{\alpha}{2\alpha-1}e^{(\alpha-1)\Delta/\lambda} + \frac{\alpha-1}{2\alpha-1}e^{-\alpha\Delta/\lambda}\right)
\]
[1702.07476], [2309.12647].

Truncated mechanisms (output range restricted to $[a, b]$) preserve the exact same RDP bounds as untruncated ones [2309.12647].

### Quantized Gaussian and Mixture Mechanisms

Quantization post-Gaussian noise further tightens RDP: for quantization to $2^b$ levels, the privacy budget $\varepsilon_q(\alpha)$ is strictly less than the standard Gaussian case and decreases with lower bit-depth [2405.10096].

Gaussian sketching ("Gaussian mixing") provides explicit, instance-adaptive RDP bounds and efficient private regression procedures. For $k$-dimensional sketches, minimum row norm $C_x$, and minimum eigenvalue $\bar{\lambda}_{\min}$,

\[
\varepsilon(\alpha) = \left[\frac{k\alpha}{2(\alpha-1)}\right]\log(1-1/\gamma) - \left[\frac{k}{2(\alpha-1)}\right]\log(1-\alpha/\gamma)
\]
where $\gamma = (\sigma^2 + \bar{\lambda}_{\min})/C_x^2$ and $1 < \alpha < \gamma$ [2505.24603].

## 5. Advanced Composition, Adaptive Strategies, and Robustness

RDP enables adaptive composition via privacy filters and odometers that support privacy budget reallocations and early stopping, ensuring global RDP/DP budgets are not exceeded [2103.01379]. These approaches give concrete gains in private SGD—e.g., higher test accuracy when using adaptive noise/batch schedules or early-stopped optimization at the same privacy loss.

PTR mechanisms can be analyzed tightly using RDP, enabling robust private learning under data corruption and local sensitivity analyses [2209.07716].

In robust interpretability, RDP is used to quantify certifiable top-$k$ robustness properties for interpretation maps, allowing provable insensitivity to adversarial perturbations of the input [2107.01561].

## 6. RDP in Generative and Synthetic Data Methods

RDP is the analytic backbone for private synthetic data generation. Adding randomness via Gaussian copulas or posterior sampling can directly satisfy $(\alpha, \varepsilon)$-RDP, as the randomness in the generative sampling "hides" individual influences [2303.17849], [1710.00892]. In deep generative models like RDP-GAN and FLIP, tight per-iteration RDP accounting yields high utility for strong privacy guarantees, supporting fairness modules and adaptive noise scaling [2007.02056], [2508.21815].

Posterior sampling mechanisms, especially in exponential families and GLMs, allow explicit control of the RDP curve by tempering likelihoods or tuning priors [1710.00892]. For synthetic data, even using non-private moments/covariance, a mechanism sampling from $\mathcal{N}(\mu, \Sigma)$ achieves $(4, 0.58)$-RDP for $n=10^7$, which can convert to standard $(4.0, 10^{-10})$-DP [2303.17849].

## 7. Extensions: Heavy Tails, Alternative Mechanisms, and Limitations

Recent work generalizes RDP analysis to heavy-tailed SDEs, establishing the first dimension-tolerant RDP guarantees in this setting by leveraging fractional Poincaré inequalities. For processes $dW_t = -\nabla R_S(W_t)dt + \sigma_\alpha dL_t^{\alpha} + \sigma_2\sqrt{2} dB_t$, the RDP bound is

\[
(\beta, \kappa)\text{-RDP} \quad \text{with} \quad \kappa = \min\left\{\frac{\beta S_g^2}{2n^2\sigma_2^2}T, \ -\ln\left(1- \frac{\gamma\beta^2S_g^2}{2n^2\sigma_2^2}\right)\right\}
\]
where $\gamma$ is the FPI constant, $S_g$ is sensitivity, and $n$ is dataset size [2511.15634].

Limitations include dependence on regularity (e.g., Poincaré or log-Sobolev) assumptions, the need for positive-definiteness in covariance, and possible looseness in finite-$n$ scenarios or for non-Gaussian mechanisms.

---

**References**  
- [1702.07476] Ilya Mironov, "Renyi Differential Privacy"  
- [1808.00087] Wang, Balle, Kasiviswanathan, "Subsampled Rényi Differential Privacy and Analytical Moments Accountant"  
- [1908.10530] Balle, Gaboardi, "Rényi Differential Privacy of the Sampled Gaussian Mechanism"  
- [2103.01379] Feldman, Zrnic, "Practical Privacy Filters and Odometers with Rényi Differential Privacy..."  
- [2309.12647] Xiang, Li, "Truncated Laplace and Gaussian mechanisms of RDP"  
- [2007.02056] Zhang et al., "RDP-GAN: A Rényi-Differential Privacy based Generative Adversarial Network"  
- [2505.24603] Ni, Nakkiran, "The Gaussian Mixing Mechanism: Renyi Differential Privacy via Gaussian Sketches"  
- [2405.10096] Liang, Song, "The Effect of Quantization in Federated Learning: A Rényi Differential Privacy Perspective"  
- [2105.05180] Girgis et al., "On the Renyi Differential Privacy of the Shuffle Model"  
- [2209.07716] Ye, Bao, "Renyi Differential Privacy of Propose-Test-Release..."  
- [2303.17849] He, Wang, "On Rényi Differential Privacy in Statistics-Based Synthetic Data Generation"  
- [2511.15634] Leleu, Moulines, "Rényi Differential Privacy for Heavy-Tailed SDEs via Fractional Poincaré Inequalities"  
- [2107.01561] Liu, Zhang, "Certifiably Robust Interpretation via Renyi Differential Privacy"  
- [2508.21815] Thomas et al., "Achieving Hilbert-Schmidt Independence Under Rényi Differential Privacy for Fair and Private Data Generation"  
- [1710.00892] Foulds et al., "Rényi Differential Privacy Mechanisms for Posterior Sampling"  
- [1907.05156] Cheu, Smith, "Amplifying Rényi Differential Privacy via Shuffling"  
- [2401.04306] Zhu, Wang, "Renyi Differential Privacy in the Shuffle Model: Enhanced Amplification Bounds"  
- [1905.09982] Gaboardi, "Hypothesis Testing Interpretations and Renyi Differential Privacy"  
- [2107.08763] Feldman, McMillan, Talwar, "Renyi Differential Privacy of the Subsampled Shuffle Model in Distributed Learning"

Source: https://www.emergentmind.com/topics/renyi-differential-privacy-rdp