Rényi Differential Privacy (RDP)

• Rényi Differential Privacy is a relaxation of differential privacy that uses Rényi divergence to measure and control privacy loss in randomized algorithms.
• It supports precise composition analyses in subsampling, adaptive mechanisms, and moments accounting, ensuring robust and modular privacy guarantees.
• RDP underpins modern private machine learning and synthetic data generation by enabling efficient noise tuning and conversion to classical (ε,δ)-DP.

Rényi Differential Privacy (RDP) is a relaxation of differential privacy formalized by Mironov (2017) that parameterizes the privacy loss of randomized algorithms using Rényi divergence, which is a moment-based generalization of the classical max-divergence used in differential privacy. RDP enables tight and modular analysis of privacy composition—particularly for mechanisms employing subsampling, adaptive composition, or moments accountants—and is now the analytic foundation for most state-of-the-art private machine learning pipelines.

1. Formal Definition and Divergence

Let $P$ and $Q$ be two probability measures defined on the same measurable space. For any order $\alpha>1$, the Rényi divergence of order $\alpha$ is

$$D_\alpha(P\|Q) = \frac{1}{\alpha-1} \log \mathbb{E}_{x\sim Q}\left[\left(\frac{P(x)}{Q(x)}\right)^\alpha\right].$$

A randomized mechanism $M$ operating on databases satisfies $(\alpha,\varepsilon)$-Rényi Differential Privacy if for every pair of adjacent datasets $D,D'$ (differing in one element):

$$D_\alpha\bigl(M(D)\|\;M(D')\bigr)\;\le\;\varepsilon.$$

As $\alpha\to1$, $D_\alpha(P\|Q)$ approaches the Kullback–Leibler divergence; as $\alpha\to\infty$, $D_\alpha(P\|Q)$ approaches the max-divergence, recovering pure $\varepsilon$-differential privacy (Mironov, 2017).

2. Composition, Conversion, and Accounting

Sequential and Adaptive Composition

If mechanisms $M_1,\ldots,M_k$ each satisfy $(\alpha,\varepsilon_i)$-RDP (possibly with different $\varepsilon_i$ but the same $\alpha$), then their k-fold (possibly adaptive) composition is $(\alpha, \sum_{i=1}^k \varepsilon_i)$-RDP (Mironov, 2017, Wang et al., 2018). This enables linear, order-preserving tracking of privacy loss under repeated or adaptive use of RDP mechanisms.

Conversion to ($\varepsilon$, $\delta$)-Differential Privacy

Any mechanism guaranteeing $(\alpha,\varepsilon)$-RDP also satisfies $$(\varepsilon+\frac{\log(1/\delta)}{\alpha-1},\ \delta)$$–differential privacy for any $0 < \delta < 1$:

$$\varepsilon' = \varepsilon + \frac{\log(1/\delta)}{\alpha-1}$$

(Mironov, 2017, Wang et al., 2018, Lécuyer, 2021).

The parameter $\alpha$ is typically tuned to optimize $\varepsilon'$ for a fixed target $\delta$.

Analytical Moments Accountant

The cumulant generating function (CGF) of the privacy loss for a mechanism $M$ is

$$K_M(\lambda) = \log \mathbb{E}_{\theta\sim M(X')}\left[\left(\frac{M(X)(\theta)}{M(X')(\theta)}\right)^{\lambda+1}\right]$$

with $\varepsilon(\alpha) = K_M(\alpha-1)/(\alpha-1)$. The moments accountant accumulates CGFs additively for composed mechanisms:

$$K_{\textrm{tot}}(\lambda) = \sum_{i=1}^k K_{M_i}(\lambda)$$

and supports tight ($\varepsilon$, $\delta$)-DP conversion via univariate convex optimization (Wang et al., 2018).

3. Amplification by Subsampling and Shuffling

Subsampling (selecting a random subset before applying a mechanism) and shuffling (randomly permuting outputs in the distributed local setting) both amplify privacy.

Subsampling

For Poisson sampling at rate $\gamma$, then applying an $(\alpha,\varepsilon(\alpha))$-RDP mechanism $M$ to the sample, the composed mechanism is $(\alpha, \varepsilon'(\alpha))$-RDP, where

$$\varepsilon'(\alpha) \le \frac{1}{\alpha - 1} \log\left(1 + \sum_{j=2}^\alpha \binom{\alpha}{j}\gamma^j c_j\right)$$

and $c_j$ are derived from the moments of $M$ (Wang et al., 2018). In the Gaussian case with sampling probability $q$ and noise $\sigma$, a tight closed form is $\varepsilon(\alpha) \le 2q^2\alpha/\sigma^2$ for valid regimes (Mironov et al., 2019).

Shuffling

Shuffling can quadratically amplify privacy in the number of records under local randomization. If $n$ users each apply an $\varepsilon_0$-LDP mechanism, then shuffling yields

$$\varepsilon_{\textrm{shuf}}(\alpha) \le \frac{1}{\alpha-1}\log\left(1+\sum_{j=2}^{\alpha} \binom{\alpha}{j} \frac{(e^{\varepsilon_0} - 1)^j}{n^{j-1}}\right)$$

(Berthier et al., 2019, Girgis et al., 2021, Chen et al., 9 Jan 2024). For large $n$ and small $\varepsilon_0$, this is $O(\varepsilon_0^2\alpha/n)$.

Shuffle model analyses now reach closed-form expressions and asymptotically optimal bounds (Chen et al., 9 Jan 2024).

4. Mechanism-Specific RDP Guarantees

Gaussian Mechanism

If $f$ has $\ell_2$-sensitivity $\Delta$, releasing $f(D) + \mathcal{N}(0, \sigma^2 I)$ gives

$$D_\alpha(\mathcal{N}(0, \sigma^2)\|\mathcal{N}(\Delta, \sigma^2)) = \frac{\alpha \Delta^2}{2\sigma^2}$$

Therefore, the mechanism is $(\alpha, \alpha\Delta^2/(2\sigma^2))$-RDP (Mironov, 2017, Mironov et al., 2019).

Laplace Mechanism

If $f$ has $\ell_1$-sensitivity $\Delta$, releasing $f(D) + \mathrm{Laplace}(\lambda)$ yields

$$D_\alpha(\mathrm{Lap}(0, \lambda)\|\mathrm{Lap}(\Delta, \lambda)) = \frac{1}{\alpha-1} \log\left(\frac{\alpha}{2\alpha-1}e^{(\alpha-1)\Delta/\lambda} + \frac{\alpha-1}{2\alpha-1}e^{-\alpha\Delta/\lambda}\right)$$

(Mironov, 2017, Fu et al., 2023).

Truncated mechanisms (output range restricted to $[a, b]$) preserve the exact same RDP bounds as untruncated ones (Fu et al., 2023).

Quantized Gaussian and Mixture Mechanisms

Quantization post-Gaussian noise further tightens RDP: for quantization to $2^b$ levels, the privacy budget $\varepsilon_q(\alpha)$ is strictly less than the standard Gaussian case and decreases with lower bit-depth (Kang et al., 16 May 2024).

Gaussian sketching ("Gaussian mixing") provides explicit, instance-adaptive RDP bounds and efficient private regression procedures. For $k$-dimensional sketches, minimum row norm $C_x$, and minimum eigenvalue $\bar{\lambda}_{\min}$,

$$\varepsilon(\alpha) = \left[\frac{k\alpha}{2(\alpha-1)}\right]\log(1-1/\gamma) - \left[\frac{k}{2(\alpha-1)}\right]\log(1-\alpha/\gamma)$$

where $\gamma = (\sigma^2 + \bar{\lambda}_{\min})/C_x^2$ and $1 < \alpha < \gamma$ (Lev et al., 30 May 2025).

5. Advanced Composition, Adaptive Strategies, and Robustness

RDP enables adaptive composition via privacy filters and odometers that support privacy budget reallocations and early stopping, ensuring global RDP/DP budgets are not exceeded (Lécuyer, 2021). These approaches give concrete gains in private SGD—e.g., higher test accuracy when using adaptive noise/batch schedules or early-stopped optimization at the same privacy loss.

PTR mechanisms can be analyzed tightly using RDP, enabling robust private learning under data corruption and local sensitivity analyses (Wang et al., 2022).

In robust interpretability, RDP is used to quantify certifiable top-$k$ robustness properties for interpretation maps, allowing provable insensitivity to adversarial perturbations of the input (Liu et al., 2021).

6. RDP in Generative and Synthetic Data Methods

RDP is the analytic backbone for private synthetic data generation. Adding randomness via Gaussian copulas or posterior sampling can directly satisfy $(\alpha, \varepsilon)$-RDP, as the randomness in the generative sampling "hides" individual influences (Miura et al., 2023, Geumlek et al., 2017). In deep generative models like RDP-GAN and FLIP, tight per-iteration RDP accounting yields high utility for strong privacy guarantees, supporting fairness modules and adaptive noise scaling (Ma et al., 2020, Hyrup et al., 29 Aug 2025).

Posterior sampling mechanisms, especially in exponential families and GLMs, allow explicit control of the RDP curve by tempering likelihoods or tuning priors (Geumlek et al., 2017). For synthetic data, even using non-private moments/covariance, a mechanism sampling from $\mathcal{N}(\mu, \Sigma)$ achieves $(4, 0.58)$-RDP for $n=10^7$, which can convert to standard $(4.0, 10^{-10})$-DP (Miura et al., 2023).

7. Extensions: Heavy Tails, Alternative Mechanisms, and Limitations

Recent work generalizes RDP analysis to heavy-tailed SDEs, establishing the first dimension-tolerant RDP guarantees in this setting by leveraging fractional Poincaré inequalities. For processes $$dW_t = -\nabla R_S(W_t)dt + \sigma_\alpha dL_t^{\alpha} + \sigma_2\sqrt{2} dB_t$$, the RDP bound is

$$(\beta, \kappa)\text{-RDP} \quad \text{with} \quad \kappa = \min\left\{\frac{\beta S_g^2}{2n^2\sigma_2^2}T, \ -\ln\left(1- \frac{\gamma\beta^2S_g^2}{2n^2\sigma_2^2}\right)\right\}$$

where $\gamma$ is the FPI constant, $S_g$ is sensitivity, and $n$ is dataset size (Dupuis et al., 19 Nov 2025).

Limitations include dependence on regularity (e.g., Poincaré or log-Sobolev) assumptions, the need for positive-definiteness in covariance, and possible looseness in finite-$n$ scenarios or for non-Gaussian mechanisms.

---

References

• (Mironov, 2017) Ilya Mironov, "Renyi Differential Privacy"
• (Wang et al., 2018) Wang, Balle, Kasiviswanathan, "Subsampled Rényi Differential Privacy and Analytical Moments Accountant"
• (Mironov et al., 2019) Balle, Gaboardi, "Rényi Differential Privacy of the Sampled Gaussian Mechanism"
• (Lécuyer, 2021) Feldman, Zrnic, "Practical Privacy Filters and Odometers with Rényi Differential Privacy..."
• (Fu et al., 2023) Xiang, Li, "Truncated Laplace and Gaussian mechanisms of RDP"
• (Ma et al., 2020) Zhang et al., "RDP-GAN: A Rényi-Differential Privacy based Generative Adversarial Network"
• (Lev et al., 30 May 2025) Ni, Nakkiran, "The Gaussian Mixing Mechanism: Renyi Differential Privacy via Gaussian Sketches"
• (Kang et al., 16 May 2024) Liang, Song, "The Effect of Quantization in Federated Learning: A Rényi Differential Privacy Perspective"
• (Girgis et al., 2021) Girgis et al., "On the Renyi Differential Privacy of the Shuffle Model"
• (Wang et al., 2022) Ye, Bao, "Renyi Differential Privacy of Propose-Test-Release..."
• (Miura et al., 2023) He, Wang, "On Rényi Differential Privacy in Statistics-Based Synthetic Data Generation"
• (Dupuis et al., 19 Nov 2025) Leleu, Moulines, "Rényi Differential Privacy for Heavy-Tailed SDEs via Fractional Poincaré Inequalities"
• (Liu et al., 2021) Liu, Zhang, "Certifiably Robust Interpretation via Renyi Differential Privacy"
• (Hyrup et al., 29 Aug 2025) Thomas et al., "Achieving Hilbert-Schmidt Independence Under Rényi Differential Privacy for Fair and Private Data Generation"
• (Geumlek et al., 2017) Foulds et al., "Rényi Differential Privacy Mechanisms for Posterior Sampling"
• (Berthier et al., 2019) Cheu, Smith, "Amplifying Rényi Differential Privacy via Shuffling"
• (Chen et al., 9 Jan 2024) Zhu, Wang, "Renyi Differential Privacy in the Shuffle Model: Enhanced Amplification Bounds"
• (Balle et al., 2019) Gaboardi, "Hypothesis Testing Interpretations and Renyi Differential Privacy"
• (Girgis et al., 2021) Feldman, McMillan, Talwar, "Renyi Differential Privacy of the Subsampled Shuffle Model in Distributed Learning"