Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds (1605.02065v1)

Abstract: "Concentrated differential privacy" was recently introduced by Dwork and Rothblum as a relaxation of differential privacy, which permits sharper analyses of many privacy-preserving computations. We present an alternative formulation of the concept of concentrated differential privacy in terms of the Renyi divergence between the distributions obtained by running an algorithm on neighboring inputs. With this reformulation in hand, we prove sharper quantitative results, establish lower bounds, and raise a few new questions. We also unify this approach with approximate differential privacy by giving an appropriate definition of "approximate concentrated differential privacy."

• The paper introduces zero-concentrated differential privacy (zCDP) to simplify privacy analysis using Rényi divergence.
• It demonstrates robust composition and postprocessing properties along with strong group privacy, leading to sharp lower bounds.
• It extends concentrated DP to an approximate framework, bridging pure and approximate differential privacy for practical applications.

Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds

Introduction

In the field of privacy-preserving data analysis, differential privacy (DP) has firmly established itself as a key framework. The concept of (pure) differential privacy, characterized by a privacy parameter $\varepsilon$, governs the privacy loss incurred during a computation involving individual data points. While DP ensures smooth degradation under the composition of multiple computations, a relaxation known as approximate differential privacy, parameterized by $(\varepsilon, \delta)$, allows for more flexible privacy guarantees by bounding the probability of exceeding a specified privacy loss $\varepsilon$ by $\delta$.

To address certain limitations and achieve sharper analyses of fundamental privacy-preserving computations, Dwork and Rothblum introduced concentrated differential privacy (CDP). CDP characterizes the privacy loss as having a small mean and being subgaussian.

Zero-Concentrated Differential Privacy (zCDP)

This paper introduces an alternative formulation to Dwork and Rothblum’s mean-concentrated differential privacy (mCDP), termed zero-concentrated differential privacy (zCDP). Unlike mCDP, zCDP uses the R\'enyi divergence to enforce that the privacy loss is concentrated around zero and resembles a Gaussian distribution with specific statistical properties.

Key Definitions and Properties

• Zero-Concentrated Differential Privacy (zCDP): A mechanism $M : \mathcal{X}^n \to \mathcal{Y}$ is $(\xi,\rho)$-zCDP if for any inputs $x$ and $x'$ differing in one element and for all $\alpha \in (1, \infty)$, $$D_\alpha(M(x) \parallel M(x')) \leq \xi + \rho \cdot \alpha$$ where $D_\alpha$ denotes the R\'enyi divergence.
• Relationship to Differential Privacy (DP): A mechanism satisfying $\varepsilon$-DP also satisfies $\left(\frac{\epsilon^2}{2}\right)$-zCDP. Conversely, $(\xi,\rho)$-zCDP implies $(\varepsilon, \delta)$-DP with $\varepsilon = \xi + 2\sqrt{\log(1/\delta)}$ for any $\delta>0$.
• Gaussian Mechanism: For a sensitivity-$\Delta$ query $q$, the Gaussian mechanism with variance $\sigma^2$ satisfies $\left(\frac{\Delta^2}{2\sigma^2}\right)$-zCDP. This demonstrates the tightness of zCDP for this mechanism.

Composition and Postprocessing

zCDP inherits differential privacy's valuable properties of composability and invariance under postprocessing. These properties simplify the analysis and usage of privacy-preserving mechanisms:

• Composition: If mechanisms $M$ and $M'$ satisfy $(\xi,\rho)$-zCDP and $(\xi',\rho')$-zCDP respectively, their composition $(M, M')$ satisfies $(\xi+\xi',\rho+\rho')$-zCDP.
• Postprocessing: For any function $f$, if $M$ satisfies $(\xi, \rho)$-zCDP, then $f \circ M$ also satisfies $(\xi, \rho)$-zCDP.

Group Privacy

zCDP extends to group privacy. Specifically, if a mechanism satisfies $(\xi,\rho)$-zCDP, then it affords $(k^2 \rho)$-zCDP for groups of size $k$, which holds universally for all data elements differing in up to $k$ entries.

Strong Numerical Results: Lower Bounds

Lower bounds derived from strong group privacy guarantees can significantly limit zCDP mechanisms. For example, answering point queries with zCDP requires significantly larger datasets compared to approximate differential privacy, illustrating the more stringent nature of zCDP.

Approximate zCDP

To circumvent the strong lower bounds of zCDP, approximate zCDP is introduced, paralleling approximate $(\varepsilon,\delta)$-DP. This relaxation retains the desirable properties of zCDP while mitigating the impact of worst-case privacy breaches by allowing a small probability of failure $\delta$.

Relation and Implications

The paper's findings underscore that zCDP simplifies and sharpens privacy guarantees. It provides a unified framework for analysing and improving privacy-preserving computations, bridging the gap between pure and approximate DP. Future work should further explore the scope and practical implications of zCDP, particularly in contexts where existing differential privacy mechanisms prove inadequate.

Conclusion

This paper delineates a refined view of concentrated differential privacy through the lens of zero-concentrated differential privacy. By leveraging R\'enyi divergence, zCDP provides tighter and conceptually cleaner privacy guarantees. It maintains core properties of differential privacy while adapting to various practical use cases, and its interplay with approximate zCDP broadens the horizon for effective privacy mechanisms. The implications of these results extend across the theoretical and practical aspects of privacy-preserving data analysis in both academia and industry.