Escaping Iterative Parameter-Space Noise: Differentially Private Learning with a Hypernetwork

Abstract: Differentially private (DP) training of neural networks is often hindered by the large amount of noise required by gradient-based methods such as DP-SGD, which repeatedly inject high-dimensional noise in parameter space throughout training. In this paper, we propose a new framework for DP learning that avoids iterative optimization in parameter space. Instead of updating the target model using privatized gradients, we employ a hypernetwork trained on public datasets to map a private dataset to the parameters of the target model. Specifically, each example is embedded into a low-dimensional representation, the embeddings are aggregated and perturbed to obtain a DP dataset embedding, and the hypernetwork generates the target model parameters from this noisy embedding. Because privacy noise is injected only once into a low-dimensional dataset representation, our approach can significantly reduce the adverse effect of noise. We theoretically show in a synthetic setting that, under a fixed privacy budget, models produced by our approach achieve higher utility than those trained with DP-SGD. Moreover, we apply our approach to LoRA fine-tuning of diffusion models and show that it achieves lower FID than LoRA models trained with DP-SGD and other public-data-guided methods.

• The paper introduces DP-DeepSets, a hypernetwork method that generates differentially private parameters by injecting noise only once into low-dimensional embeddings.
• It leverages a transformer-based decoder and mean pooling to capture inter-layer dependencies while ensuring rigorous privacy guarantees.
• Empirical evaluations on LoRA fine-tuning for diffusion models demonstrate that DP-DeepSets achieves lower FID scores than DP-SGD under strict privacy budgets.

Differentially Private Learning with Hypernetwork Parameter Generation

Motivation and Problem Setting

Differentially private (DP) training of deep neural networks is a pivotal requirement for learning from sensitive data, with DP-SGD being the dominant paradigm. However, the repeated injection of high-dimensional Gaussian noise into gradients during SGD optimization, especially in small-data regimes such as LoRA fine-tuning, significantly degrades model utility. The noise scale grows with both the parameter dimension and the number of optimization steps, creating a fundamental constraint that cannot be bypassed by traditional gradient-based DP methods. This paper proposes an alternative approach: directly generating differentially private parameters via a trained hypernetwork, thereby avoiding iterative noise injection in parameter space.

Framework: DP-DeepSets Hypernetwork Architecture

The introduced DP-DeepSets framework leverages a parameter-generating hypernetwork trained entirely on public datasets. Each data point in the private set is first embedded with a shared MLP, producing low-dimensional representations which are then clipped in $L^2$ norm. Dataset-level aggregation proceeds via mean pooling, followed by Gaussian perturbation in accordance with DP guarantees.

The entire parameter-generation pipeline thus injects privacy noise only once—into the low-dimensional dataset embedding—leading to a substantial reduction in utility loss. Post-processing properties of DP ensure that the final target network parameters inherit the same privacy guarantee.

Figure 1: DP-DeepSets workflow—embedding, clipping, mean pooling, and noise injection together yield a DP-compliant dataset representation for parameter generation.

The parameter-generation function $\phi_2$ employs a transformer-based decoder with context-token conditioning and query-token partitioning across the target parameter vector, enabling modeling of inter-layer dependencies and flexible parameter generation.

Figure 2: Transformer-based decoder architecture of DP-DeepSets, leveraging cross-attention between dataset embedding and parameter tokens.

Hypernetwork Training and DP Guarantee

Hypernetwork training uses only public datasets to learn a global mapping from dataset representations to target model parameters, minimizing dataset-wise empirical risk on public data batches. During inference, Gaussian noise is added to dataset embeddings for DP compliance. The DP budget is consumed exclusively during private-data inference; hypernetwork training remains privacy-neutral.

Figure 3: Training procedure of DP-DeepSets hypernetwork—end-to-end optimization over multiple public datasets establishes dataset-to-parameter mapping.

A rigorous DP guarantee is established by applying the Gaussian mechanism to the mean embedding, producing $(\epsilon,\delta)$-DP under appropriate noise scaling.

Theoretical Analysis: Linear Regression Case Study

The paper presents a comparative theoretical analysis between DP-DeepSets and DP-SGD in a linear regression scenario with low intrinsic dimensionality. DP-DeepSets exploits task structure, embedding datasets in a low-dimensional space and injecting noise only once. The derived upper bound on estimation error demonstrates a trade-off governed by embedding dimension $k$: larger $k$ reduces low-dimensional approximation error but increases noise-induced error.

For DP-SGD, the lower bound reveals that repeated noise injection in high-dimensional parameter space precludes outperforming DP-DeepSets, especially in regimes with high parameter dimension and limited training samples.

Quantitative Results: In the high-dimensional regime ($d\to\infty$), DP-DeepSets' error scales as $$O\left(\frac{d}{N}+\left(\frac{1}{\epsilon N}\right)^{\frac{2\gamma-1}{\gamma+1}}\right)$$, substantially outperforming DP-SGD whose error remains $\Omega(1)$ under fixed privacy budgets.

Empirical Evaluation: LoRA Fine-Tuning for Diffusion Models

Empirical evaluation focuses on LoRA fine-tuning of a diffusion model (ImageNet64) using small private datasets (CIFAR-10). DP-DeepSets is trained on 30,000 public datasets (each a $k$-NN set of 128 CLIP-embedded images), generating LoRA parameters for 100 distinct private datasets.

FID Score Comparison: Across varying values of $\epsilon$, the DP-DeepSets method achieves consistently lower FID scores compared to DP-SGD and public-data-guided baselines (PDA-DPMD and public-data finetuning followed by DP-SGD), especially in the stringent privacy regime ($\phi_2$0). DP-SGD exhibits rapid utility collapse as $\phi_2$1 decreases, whereas DP-DeepSets maintains performance up to $\phi_2$2.

Figure 4: Left—CIFAR-10 samples generated using DP-DeepSets. Right—FID as a function of DP $\phi_2$3; DP-DeepSets shows significantly lower FID across all privacy budgets.

DP-DeepSets also demonstrates robust generalization under distribution shift: transferring hypernetwork training from ImageNet64 to CIFAR-10 preserves utility and maintains superiority over DP-SGD even for small private datasets.

Practical Implications and Theoretical Impact

This framework's core implication is a decoupling of DP noise scaling from parameter space dimensionality, achieved by embedding private datasets in low-dimensional manifolds and performing one-shot noise injection. Such an approach is well-suited for few-shot or small-batch scenarios where traditional DP-SGD is untenable.

The paradigm also paves the way for meta-learning schemes, exploiting hypernetwork "learning-to-learn" on public data to effectively encode high-dimensional task structure. Theoretically, it resolves DP-SGD's noise scaling bottleneck, demonstrating that direct parameter generation architectures can outperform gradient-based DP in both utility and privacy/utility trade-off.

Future Directions

Extending DP-DeepSets to other model modalities (e.g., language) and diverse tasks (classification, sequence generation, etc.) is a critical next step. Additionally, scaling analysis for larger dataset sizes and refining hypernetwork architectures for more complex parameter spaces remain open research questions. Understanding the limits of low-dimensional embedding and its interaction with privacy/utility trade-off is also promising.

Conclusion

The paper introduces a principled framework for differentially private learning using hypernetwork parameter generation, escaping the constraints of iterative, high-dimensional noise injection. DP-DeepSets demonstrates both theoretical and empirical superiority in key regimes, substantiating its viability as an alternative to DP-SGD when faced with high parameter dimension and limited private data. The practical and theoretical advancements presented strongly motivate widespread adoption and further extension of this paradigm (2606.26772).