Scalable Differentially Private Data Compression via Diffusion and Stochastic Codes

Abstract: The ever-increasing collection of personal data has created mounting pressure to develop technologies that protect sensitive aspects of individual identity. Differential privacy (DP) provides a principled framework with strong formal guarantees and has already achieved practical success. However, releasing high-dimensional data, such as images, has remained elusive: releasing uncompressed privatized data requires significant storage. At the same time, no effective data compression scheme exists that can compress high-resolution data with privacy guarantees. We address this challenge with DP-DiPP, a compression pipeline that combines stochastic codes with diffusion models. DP-DiPP is highly flexible: the practitioner has direct control over the compression rate-privacy-utility tradeoff. As the theoretical backbone, we extend the Poisson private representation (PPR) to encode the outputs of privacy mechanisms. We then combine it with DiffC, a diffusion-based lossy data compression method, to obtain a differentially private image compressor. Our experiments on privatized image classification on CIFAR-10 demonstrate that DP-DiPP significantly outperforms the baseline, achieving a 10-30 times better compression while retaining comparable privacy guarantees and utility.

• The paper presents DP-DiPP, a novel framework that interleaves stochastic encoding and diffusion-based compression to achieve provable pure differential privacy.
• It extends Poisson Private Representation with a step-limited approximation, balancing computational efficiency and precision while reducing bitrate by 10–30× at fixed privacy levels.
• The approach employs moment-matched Laplace diffusion, ensuring strong privacy and maintaining downstream utility, as demonstrated on tasks like CIFAR-10 classification.

Scalable Differentially Private Data Compression via Diffusion and Stochastic Codes

Introduction and Problem Statement

This work investigates the intersection of data compression and differential privacy (DP), focusing on the nontrivial regime of high-dimensional data (notably images) where the storage and communication costs grow prohibitively when using traditional privatization pipelines. Conventional approaches either privatize data and then compress—resulting in poor rates due to the inflated randomness—or rely on non-private lossy compressors, undermining privacy. The authors address this by introducing DP-DiPP, a framework that synergistically combines stochastic codes and diffusion-based lossy compression, establishing formal privacy certificates while maintaining practical utility and storage efficiency (2607.03392).

Theoretical Contributions

Extension of Poisson Private Representation (PPR)

A central theoretical advance lies in the generalization of the Poisson private representation (PPR). Classic PPR provides an LDP-compliant stochastic compression code but is computationally intractable for high-KL scenarios such as image data—the runtime scales as the essential supremum of the density ratio between the mechanism and proposal. The authors introduce an “approximate” or “step-limited” PPR which limits the number of proposals examined, trading off negligible approximation error and fixed runtime for small total variation distance from the ideal mechanism. The DP properties are maintained: encoding a pure $\epsilon$-LDP mechanism with parameter $\alpha > 1$ yields a $2\alpha\epsilon$-LDP certificate.

Moment-Matching Laplace Diffusion for Pure LDP

Classic diffusion-based lossy compression (e.g., DiffC, based on Gaussian denoisers) cannot realize pure LDP. The authors approximate the conditional Gaussians in the denoising process by moment-matched Laplace distributions, allowing each denoising step to be interpretable as a pure LDP mechanism under the $\ell_1$ metric. The Laplace mechanism’s density ratios are naturally bounded, essential for feasible PPR deployment. Empirically, this Laplace approximation incurs negligible distortion compared to the Gaussian baseline, but with rigorously quantifiable privacy.

Joint Privatization and Compression

The DP-DiPP code does not simply compose privatization with lossless compression. By interleaving stochastic encoding and privacy mechanisms via the denoising path of diffusion models, the framework achieves near-optimal trade-offs among rate, privacy, and utility. Its application to the CIFAR-10 classification task with privatized images shows superior efficiency; standard privatize-then-compress approaches require 10–30× higher rates for equal privacy/utility.

Algorithmic Pipeline

• Modification of DiffC: The denoising process is rewritten to use moment-matched per-step Laplace noise, ensuring each step satisfies a bounded, explicitly computed pure LDP guarantee (per channel and composed per pixel).
• Step-Limited PPR: At each denoising/compression step, samples are encoded using the step-limited PPR, leveraging a custom CUDA kernel for parallel proposal evaluation.
• Optimal Schedule Selection: Compression schedules are selected via a shortest-path optimization over stepwise privacy-utility-resource costs, tuned per desired privacy $\epsilon_{\text{total}}$ as a constraint.

  Figure 1: Per $\epsilon$ distribution of the standardized absolute mean deviation $\delta$, over all time steps, images, and latent space dimensions, revealing the practical regime for moment-matching.

Experimental Results

Experiments are conducted on privatized image compression and classification over CIFAR-10, comparing DP-DiPP, privatize-then-compress (Laplace + PNG), and the original DiffC baseline. Key findings include:

• At fixed privacy (measured by per-pixel $\epsilon$-LDP), DP-DiPP achieves classification accuracy on par with the non-private data, but with a 10–30× lower bitrate than the baseline method.
• Switching from the original Gaussian mechanism to the moment-matched Laplace (necessary for pure LDP) doubles the bitrate in practice, but does not degrade downstream utility; this factor is explained theoretically by the relative entropy increase when moment-matching (see Section \ref{sec:factor_two_increase_explanation}, also corroborated by the empirical distribution in Figure 1).

The composite privacy guarantee of DP-DiPP remains pure LDP even after the entire denoising/compression path, thanks to the per-step composition of PPR and the Laplace mechanism.

Discussion and Implications

The proposed framework closes the gap between the communication-privacy-utility trilemma for high-dimensional data, addressing a previously unresolved practical bottleneck in DP. It provides tunable control over all resource axes—runtime is linear in the step-budget, the privacy parameter $\alpha$ is explicit, and rate/utility targets can be optimized post hoc.

Practically, DP-DiPP is well-suited for privacy-preserving synthetic dataset release, edge learning scenarios, or any multi-party setting requiring strong privacy with bandwidth constraints. The reliance on stochastic codes and progressively denoised diffusion models makes it extensible to other generative frameworks, including latent and foundation models as they become standard in compression and generative tasks.

Theoretically, natural extensions include handling $(\epsilon, \delta)$-LDP or R\'enyi-DP generalizations for centralized or federated learning protocols. The step-limited PPR analysis opens avenues for error-controlled approximate channel simulation and mechanism design where runtime/cost are explicit optimization objectives.

Conclusion

This work provides an explicit, efficient, and provably differentially private pipeline for high-dimensional data compression based on diffusion models and stochastic codes. The DP-DiPP method enables significant compression gains over sequential privatize-then-compress schemes, with no degradation in downstream utility. The framework bridges a theoretical and practical gap in privacy-preserving ML, and its constructs can be generalized to more complex data modalities and advanced privacy notions.

• "Scalable Differentially Private Data Compression via Diffusion and Stochastic Codes" (2607.03392)
• For theoretical background: "Universal exact compression of differentially private mechanisms" [liu2024universal], "Channel simulation: Theory and applications to lossy compression and differential privacy" [li2024channel], "Lossy compression with Gaussian diffusion" [theis2022lossy], "Lossy compression with pretrained diffusion models" [vonderfecht2025lossy], "Breaking the communication-privacy-accuracy trilemma" [chen2020breaking].