--- title: Differentially Private Training url: https://www.emergentmind.com/topics/differentially-private-training type: topic --- # Differentially Private Training Differentially private training is a framework for learning models from sensitive data while providing formal, quantifiable privacy guarantees for each individual in the training set. Differential privacy (DP) ensures that the presence or absence of any single data point only minimally influences the model, typically by injecting calibrated noise at strategic points in the optimization workflow. Over the last decade, differentially private training has become the de facto standard in privacy-preserving deep learning and has been deployed across domains such as large-scale vision, language, recommendation, and federated learning. ## 1. Formal Foundations and Privacy Accounting A randomized algorithm $\mathcal{M}$ is said to provide $(\epsilon, \delta)$-differential privacy if for any pair of neighboring datasets $D, D'$ (differing in one example), and for all $S \subseteq \text{Range}(\mathcal{M})$, $$ \Pr[\mathcal{M}(D) \in S] \leq e^\epsilon \Pr[\mathcal{M}(D') \in S] + \delta. $$ The canonical privacy mechanism is the Gaussian Mechanism, which adds noise $\mathcal{N}(0, \sigma^2 S^2 I)$ to a function $f$ with $\ell_2$-sensitivity $S$. In deep learning, the per-example gradient is clipped to norm $C$, setting $S \leq C$, and noise is injected according to the above distribution [2201.12328]. Privacy loss per iteration is amplified by subsampling batches and accumulated over $T$ steps via composition bounds. Modern procedures use Rényi Differential Privacy (RDP) or moments accountant techniques for tight accounting, converting the final RDP cost to $(\epsilon, \delta)$ with $$ \epsilon = \min_{\alpha > 1} \left\{ \sum_{t=1}^T \epsilon_t(\alpha) + \frac{\ln(1/\delta)}{\alpha-1} \right\} $$ [2201.12328, 1812.06210]. ## 2. Core Methodologies in Deep Learning ### DP-SGD Baseline The fundamental algorithm underlying most differentially private training is DP-SGD (Differentially Private Stochastic Gradient Descent) [Abadi et al., 2016]: 1. Compute per-example gradients $g_i = \nabla \ell(\theta; x_i)$ for each example in the minibatch. 2. Clip each $g_i$ to norm $C$: $$ \bar{g}_i = g_i / \max(1, \|g_i\|_2 / C) $$ 3. Aggregate and add Gaussian noise: $$ \hat{g} = \frac{1}{N}\sum_i \bar{g}_i + \mathcal{N}(0, \sigma^2 C^2 I) $$ 4. Update parameters using a standard optimizer: $\theta \leftarrow \theta - \eta \hat{g}$. This approach is directly applicable at scale, but incurs significant accuracy loss in large models or with tight privacy budgets [2201.12328, 2205.02973]. ### Extensions and Alternatives - **Direct Feedback Alignment (DP-DFA):** Replaces backprop with layerwise error propagation via fixed random matrices, enabling tighter sensitivity bounds and improved utility due to earlier clipping [2010.03701]. - **Sharpness-Aware Minimization (DP-SAT):** Steers optimization to flat minima, mitigating distortion from clipping and noise, and achieves better privacy-utility trade-offs without extra privacy cost per iteration [2306.05651]. - **Semi-Sensitive Feature DP:** When some features are public, noise injection is confined to the private subset, yielding improved empirical accuracy compared to full DP or label-DP-only baselines [2401.15246]. - **Heterogeneous Noise Allocation (DP-Hero):** Adapts per-coordinate noise scales using spectral information from previously trained models, improving convergence and utility, especially at high noise [2409.03344]. ## 3. DP Training in Specialized Architectures and Modalities ### Neural Networks at Scale - **ResNets and Convolutional Models:** Novel normalization strategies such as ScaleNorm (GN after each residual addition) alleviate instability from DP noise, enabling state-of-the-art top-1 accuracy (82.5% on CIFAR-10 at $\varepsilon=8$) [2203.00324]. - **Transformers and Transfer Learning:** Pre-training on massive public corpora (e.g., JFT-300M) and DP fine-tuning of only the last (classification) layer via a single noisy full-batch step yields >81% ImageNet accuracy at $\varepsilon=10$, closing much of the non-private gap while dramatically reducing computational cost [2205.02973]. ### Federated and Split Learning - **Federated News Recommendation:** Projecting user embeddings onto low-dimensional public bases, perturbing only the combination coefficients, and applying label permutation yields a substantially lower utility loss than naive DP on full gradients [2204.08146]. - **Split Learning:** Perturbing the final hidden representation gradients in their critical projection direction enables transcript-level $(\varepsilon,0)$-DP across embeddings, gradients, and model updates, eliminating label leakage risks [2203.02073]. ### Graph Neural Networks - **Random Walk Subgraph Sampling:** Partitioning the training graph into disjoint subgraphs and treating each as an independent DP-SGD sample restores standard $2C$ sensitivity, permitting deeper GNNs and large-batch training with competitive accuracy at $\epsilon \leq 8$ [2301.00738]. ### Generative Modeling - **Generative Models:** Non-adversarial optimal transport objectives (Sinkhorn divergence) enable DP generative models to avoid the instability of DP-GANs and maintain robust downstream utility under $(\varepsilon=10,\,\delta=10^{-5})$ [2111.01177]. Retrieval-augmented DP diffusion models leverage public trajectory knowledge bases to drastically reduce the number of noisy privatized steps and outperform previous DP-diffusion approaches in both sample quality and computational efficiency [2502.12794]. - **Private Language Modeling:** Fine-tuning public language models (e.g., trained on Brown Corpus) on private data under DP-SGD produces practical $(\varepsilon=10)$ guarantees while maintaining nontrivial quality, which is not possible for DP-from-scratch runs [2009.05886]. - **Latent Dirichlet Allocation:** Both centralized DP (with Laplace noise on sufficient statistics) and local DP (randomized response over document words) have been developed to privatize LDA training; online and streaming versions utilize Bayesian denoising for improved utility [2010.04391]. ## 4. Utility-Privacy Trade-offs and Predictive Multiplicity Differentially private training mechanisms introduce a characteristic trade-off: decreasing $\varepsilon$ (increasing privacy) monotonically increases the test error and predictive multiplicity, where predictive multiplicity quantifies the chance that retraining from scratch will yield highly variable predictions for the same input [2302.14517]. Empirical studies demonstrate that for many points, particularly at small $\varepsilon$, the model's output is essentially randomized, raising equity and justifiability issues for high-stakes applications. Key empirical recommendations include: - Use the highest tolerable privacy parameter for critical use cases; - Estimate and report per-input and group-level multiplicity (disagreement rates); - Where possible, aggregate over multiple DP-trained models to reduce arbitrariness, accepting additional privacy budget cost [2302.14517]. ## 5. Privacy Analysis Techniques and Implementation Practices Privacy loss must be tightly accounted for across iterations and across multiple vector-valued aggregates (such as layerwise gradients). Modern implementations employ modular architectures that separate training logic, privacy mechanism (clipping and noise), and post-training privacy accounting (e.g., via a PrivacyLedger and accountant) [1812.06210]. Standard best practices across diverse domains include: - Prefer large batch sizes to amplify privacy via subsampling; - Replace batch normalization with per-example (e.g., group) normalization compatible with per-example gradient clipping [2201.12328]; - Pre-train on public (non-sensitive) data, then fine-tune privately to minimize the number of costly DP steps [2205.02973, 2009.05886]; - Carefully tune clipping norm $C$ and optimizer step size $\eta$ jointly to maintain learning stability under DP noise; - Track total privacy loss with RDP- or moments accountant-based tools, and optimize utility through hyperparameter sweeps conducted on disjoint public data [1812.06210, 2201.12328]. ## 6. Frontiers, Limitations, and Open Challenges Differentially private training has reached maturity in tabular, image, and text tasks, yet several research frontiers and practical obstacles remain: - **High-Utility Large-Scale DP:** Even with all best practices, a non-negligible accuracy gap remains in large models at strict $\varepsilon$ targets (e.g., $\sim20$–$30\%$ loss vs non-private on ImageNet at $\varepsilon=10$) [2201.12328]. Highly overparameterized models under DP-SGD lose more utility than smaller, well-regularized counterparts [2205.02973]. - **Specialized Mechanisms:** Novel architectures (e.g., DP-KAN, which uses Kolmogorov-Arnold decomposition and univariate splines) can match standard MLPs in degradation profile, suggesting rich structure to exploit for DP-under-architectural design [2407.12569]. - **Adaptive Noise and Heterogeneous Mechanisms:** Guidance-driven noise scaling (DP-Hero) and block coordinate approaches with adaptive/decaying noise (under the hidden-state assumption) allow tighter privacy for the same utility [2409.03344, 2407.08233]. - **Auditing and Explainability:** Documenting and communicating predictive multiplicity is necessary for equitable deployment in consequential domains [2302.14517]. - **Computation and Memory:** Efficient vectorized per-example gradient computation, virtual batching, and model partitioning are critical to scaling DP training, especially when hardware is limited. A central goal for ongoing research is closing the remaining privacy-utility gap for deep models—through architecture-aware privacy amplification, federated composition, and algorithm-specific sensitivity reductions—such that state-of-the-art non-private accuracy is achievable at tight DP budgets. --- **References** - [2107.14582] NeuralDP Differentially private neural networks by design - [2302.14517] Arbitrary Decisions are a Hidden Cost of Differentially Private Training - [2010.03701] Differentially Private Deep Learning with Direct Feedback Alignment - [2401.15246] Training Differentially Private Ad Prediction Models with Semi-Sensitive Features - [2203.00324] Differentially private training of residual networks with scale normalisation - [2111.01177] Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence - [2009.05886] Differentially Private Language Models Benefit from Public Pre-training - [2502.12794] RAPID: Retrieval Augmented Training of Differentially Private Diffusion Models - [2306.05651] Differentially Private Sharpness-Aware Training - [2204.08146] PrivateRec: Differentially Private Training and Serving for Federated News Recommendation - [2301.00738] Training Differentially Private Graph Neural Networks with Random Walk Sampling - [2407.12569] DP-KAN: Differentially Private Kolmogorov-Arnold Networks - [2409.03344] Revisiting Privacy-Utility Trade-off for DP Training with Pre-existing Knowledge - [2201.12328] Toward Training at ImageNet Scale with Differential Privacy - [2010.04391] Latent Dirichlet Allocation Model Training with Differential Privacy - [1812.06210] A General Approach to Adding Differential Privacy to Iterative Training Procedures - [2205.02973] Large Scale Transfer Learning for Differentially Private Image Classification - [2203.02073] Differentially Private Label Protection in Split Learning - [2407.08233] Hidden State Differential Private Mini-Batch Block Coordinate Descent for Multi-convexity Optimization Source: https://www.emergentmind.com/topics/differentially-private-training