PPDS System: Privacy-Preserving Data Synthesis

• PPDS system is a framework for synthetic data generation that enforces (ε, δ)-differential privacy to protect sensitive information.
• It standardizes the process through a two-stage pipeline of deconstruction and reconstruction using both statistical and deep learning models.
• The system is applied to various data types, achieving robust benchmarks via methods like DP-SGD and feature alignment techniques.

A PPDS system, within the context of privacy and synthetic data, refers to "Privacy-Preserving Data Synthesis," a research area and set of methodologies whose goal is to generate synthetic datasets that maximize data utility while enforcing rigorous differential privacy guarantees. The synthesis process integrates both formal privacy mechanisms and advanced machine learning—or statistical—models to prevent the disclosure of sensitive information about individuals in the source dataset, even to adversaries with side information. This overview presents the theoretical underpinnings, methodological taxonomy, algorithmic strategies, benchmarking, and current research challenges in PPDS systems as compiled in recent surveys and systematizations (Hu et al., 2023).

1. Formal Foundations of Privacy-Preserving Data Synthesis

The core guarantee in PPDS systems is typically expressed as (ε, δ)-differential privacy (DP), which provides a quantifiable upper bound on the probability that a single individual's data can influence the synthetic output. Formally, for a randomized mechanism $M$ and neighboring datasets $D, D'$, (ε, δ)-DP is defined as:

$$\forall\,D∼D',\,\forall\,S:\quad \Pr[M(D)\in S]\le e^{\varepsilon}\Pr[M(D')\in S]+\delta$$

where $\varepsilon$ is the privacy loss parameter (smaller is stricter), and $\delta$ is an additive slack accounting for rare failures.

A fundamental trade-off exists: tighter privacy (lower $\varepsilon$) requires greater noise, which reduces the fidelity or utility of the synthetic dataset. As a result, PPDS systems are evaluated along the privacy–utility frontier, balancing formal guarantees against practical performance.

2. Unified Methodological Framework ("Master Recipe")

Recent systematizations formalize PPDS synthesis as a two-stage pipeline (Hu et al., 2023):

• Stage 1: Deconstruction
  • Model the original data distribution $P$ or summary statistic $f(D)$.
  • Extract a compact data representation $R$ (low-order marginals, copula parameters, random projections, or deep embeddings).
  • Apply a differentially private mechanism (Gaussian, Laplace, Exponential Mechanism, or DP-SGD) to obtain a sanitized representation $\tilde{R}$.
• Stage 2: Reconstruction
  • Generate synthetic data $D, D'$0 by sampling from $D, D'$1 via statistical fitting, data generation, or feeding noise through a privatized generative model.

This standardization enables systematic comparison and modular design.

3. Taxonomy of PPDS Techniques

PPDS methods are categorized according to their backbone modeling strategies:

Category	Subtypes	Main Mechanisms
Statistical	Bayesian Networks, Markov Random Fields, Copulas, Projections	Classic DP mechanisms
Deep Learning	Autoencoders/VAEs, GANs, Optimal Transport, MMD/Feature Alignment, Diffusion/Score-based	DP-SGD, PATE, OT, MMD

Statistical Methods

• Marginal-Based: Bayesian networks such as PrivBayes factorize the data and privatize low-order marginals via the Exponential Mechanism.
• Copula-Based: Gaussian and vine copulas fit joint distributions by sanitizing correlation matrices (e.g., private Pearson/Kendall τ).
• Projection-Based: Johnson-Lindenstrauss projections, compressive sensing, or principal component analysis (PCA) using privatized statistics.
• Query-Based: MWEM, DualQuery iteratively answer and reconstruct marginals.

Deep Learning–Based Methods

• Autoencoding: Differentially private variational autoencoders or AEs use privatized latent representations.
• GANs and Minimax: Differential privacy is enforced via DP-SGD on discriminator updates or PATE aggregation (PATE-GAN, G-PATE).
• Optimal Transport (OT): DP-Sinkhorn employs the Sinkhorn divergence with privatized loss function gradients.
• Feature Alignment (MMD): DP-MERF injects noise into the estimated mean in random/Fourier/Hermite feature space, leveraging maximum mean discrepancy objectives.
• Score-based/Stochastic Simulation: DPGEN uses differentially private score-matching in SDE-generated models.

4. Algorithmic and Practical Details

Differential Privacy Implementations

• Statistical models use Laplace or Gaussian noise calibrated to the global sensitivity of sufficient statistics.
• Deep learning models routinely utilize DP-SGD, where each gradient update is individually clipped and then noised:

For a mini-batch of gradients $D, D'$2, clipped to norm $D, D'$3, generate a noisy gradient:

$D, D'$4

• Feature alignment: A single-shot perturbation of the feature mean (e.g., in DP-MERF) achieves DP without repeated privacy loss composition.

Data Synthesis and Sampling

• Statistical graphical models: Generate samples via ancestral or Gibbs sampling, using privatized conditional or joint distributions.
• Copula-based: Sample from the privatized joint copula and invert through noisy marginals.
• Deep Generative Models: Generate data by forward passes conditioned on noise, applying post-hoc privacy checks if needed.

5. Empirical Benchmarks and Comparative Results

Benchmarking on canonical datasets (MNIST, Fashion-MNIST) shows strong performance for feature alignment methods:

• DP-MERF (Feature alignment via MMD and random features):
  • Maintains ≈70% accuracy for ε=1 and ≈50% for ε=0.2 (private setting).
  • Outperforms DP-GAN, DP-CGAN, GS-WGAN (which drop below 30% at ε≤1).
  • Retains classification features well even at strict privacy budgets (ε=0.2), explained by non-compositional (one-shot) feature sanitization and convex objectives.
• DP-Sinkhorn/DPGEN perform well at moderate privacy but degrade sharply at stringent privacy levels.

This hierarchy is reflected across image and table data, though the optimal method may depend on domain (tabular, image, trajectory, or graph).

6. Method Selection and Practical Guidelines

Method selection in PPDS is scenario- and data-dependent:

• Tabular, structured: Use statistical models (PrivBayes, PrivMRF) for known low-order correlations.
• Mixed/General-purpose: DP-MERF and PATE-GAN are recommended.
• Trajectory/graph: Prefix-tree, OptaTrace, dK-series, or C-AGM.
• Images: DP-MERF for classification relevance; DP-Sinkhorn for fidelity.

Auxiliary safeguards include logging, privacy accounting (Moments Accountant or Rényi DP), and sanity-checking via membership inference attacks.

7. Open Problems and Ongoing Research

Several open questions remain in PPDS:

• Statistical models: Extending to finer constraints (functional dependencies, denial constraints), and joint modeling of mixed data types.
• Deep models: Designing DP-aware neural architectures (via neural architecture search), developing task-specific cost functions (beyond standard L₁/L₂ in OT).
• New domains: Synthesis for high-res visual or sequential data, LLMs, and robust handling of video/audio/text.
• Privacy auditing and attacks: Advanced experimental privacy auditing (e.g., synthetic data poisoning), strong linkage attacks, and diversified adversary models.
• Theory: Context-sensitive privacy metrics and formal utility–privacy benchmarks.

There is ongoing interest in achieving a consensus on evaluation standards and compositional privacy guarantees, and in developing rapid, one-shot (non-compositional) methods that maintain utility at small ε.

---

The PPDS system is the central mechanism underlying privacy-preserving synthetic data generation, formalized through (ε, δ)-differential privacy. Contemporary research delineates the area via a unified recipe, divides solutions into statistically motivated and deep learning–based methods, and empirically demonstrates that feature alignment approaches such as DP-MERF provide robust, all-purpose performance across modalities and privacy budgets (Hu et al., 2023). The field is rapidly evolving to address new data types, adversarial threats, and requirements for formal utility measurement.