Privacy-Preserving Personalized Federated Learning

• PP-PFL is a decentralized learning paradigm that integrates rigorous privacy measures (such as DP, MPC, and encryption) with personalized model optimization for diverse client distributions.
• It employs methods like stacked generalization, clustered personalization, and hybrid shared/personal head models to adapt effectively to non-IID data while preserving privacy.
• Empirical results demonstrate that PP-PFL can boost model fairness and accuracy, though privacy guarantees may incur a slight performance trade-off.

Privacy-Preserving Personalized Federated Learning (PP-PFL) refers to a class of decentralized machine learning techniques designed to simultaneously achieve user-level personalization and strong privacy guarantees. Instead of transmitting raw data from participating clients to a central server, as in classical machine learning, PP-PFL keeps sensitive data local and leverages formal privacy-preserving mechanisms—most notably, differential privacy (DP), secure multi-party computation (MPC), and encryption. In addition, PP-PFL targets individual model adaptation to heterogeneous client distributions by adopting multi-model or personalized optimization strategies. A defining characteristic of modern PP-PFL frameworks is the tight coupling of privacy, personalization, and scalability, spanning a spectrum from gradient- or model-privacy to group-level, instance-level, or metric-based privacy.

1. System Models and Privacy Foundations

Central to PP-PFL is the federated learning (FL) system architecture, where a set of distributed clients (users, devices, silos) each maintain local data $D_i$ and participate in collaborative model training via message passing rather than centralized data pooling. Typical federation modalities include:

• Horizontal FL, where all clients observe the same feature space but disjoint samples.
• Vertical FL, where clients share samples (IDs or entities) but observe different features.
• Hybrid FL, a combination of both partitions.

Privacy threats encompass a semi-honest (honest-but-curious) server, possibly adversarial or colluding clients, and potential external attackers. The core privacy objectives are:

• Preventing any party from reconstructing a client’s raw data.
• Limiting information about a client’s data that can be inferred from exchanged models, gradients, or model outputs.
• Enabling formal guarantees, such as $(\varepsilon,\delta)$-differential privacy or group/metric privacy, for local updates or exchanged partial models.

PP-PFL extends traditional FL privacy by integrating:

• Instance-level DP: Ensuring that single data samples have limited impact on any exchanged update, using Gaussian or Laplace mechanisms for noise addition (Cooper et al., 30 Jan 2025, Bose et al., 2023).
• Local (client-level) DP: Enforcing privacy guarantees at the client granularity, crucial in cross-silo settings (Tran et al., 2023).
• Metric $d$-privacy: Providing variable levels of indistinguishability based on the "closeness" of data points or updates in a specified metric space, allowing tunable privacy composition and group privacy guarantees (Galli et al., 2022).
• Secure Aggregation/MPC: Employing cryptographic protocols to ensure the server only accesses aggregated, not individual, model contributions (Khan et al., 2024).

2. Personalization Methodologies in PP-PFL

PP-PFL systems implement a variety of personalized learning strategies:

• Stacked Generalization: Clients exchange privacy-preserving base models. Each client constructs a personalized meta-model using the predictions of all base models (local and remote) as input features, typically training on a held-out private set. Privacy guarantees are enforced by applying DP or encrypting model parameters before sharing. Contributions of each client to others’ personalization can be quantified using variable-importance measures or Shapley values (Cantu-Cervini, 2024).
• Clustered Personalization: Clients are dynamically grouped into clusters based on distributional similarity (e.g., via Population Stability Index), with each cluster training a local global model. Privacy is preserved since only label counts or sanitized statistics are exchanged, and clustering is performed using features robust to raw data reconstruction (Jimenez-Gutierrez et al., 23 Dec 2025, Galli et al., 2022).
• Embedding-based Personalization: Clients maintain private embeddings encoding user-specific properties. A shared backbone and a small set of sub-population or global heads are learned, with per-user embeddings never transmitted; privacy is preserved via gradient clipping and DP on the shared parameters. Sub-population assignments are determined via clustering on the embedding space with structures such as SOM or prototype matching (Silva et al., 2022).
• Hybrid Shared/Personal Head Models: Each client has a shared "backbone" and a personalized head, with only the backbone or a portion thereof participating in federated aggregation. DP is employed on the backbone updates, and the personalized head never leaves the device (Cooper et al., 30 Jan 2025, Bose et al., 2023, Di et al., 2023).
• Meta-Learning with DP Synthetic Data: In cross-silo scenarios, clients generate differentially private synthetic data via DP-GANs and participate in federated meta-optimization using these surrogates, followed by local fine-tuning on the real private data (Tran et al., 2023).

3. Privacy-Preserving Mechanisms

State-of-the-art PP-PFL frameworks adopt one or more of the following privacy-preserving primitives:

• Differential Privacy: Each local update is clipped to a fixed norm and noised using either Gaussian or Laplace distributions, with noise calibrated to per-client or per-update sensitivity. Privacy accounting tracks total $(\varepsilon,\delta)$ via advanced composition or moments accountant across rounds (Cooper et al., 30 Jan 2025, Bose et al., 2023, Silva et al., 2022).
• Metric $d$-Privacy: Instead of uniform indistinguishability, the privacy mechanism’s protection scales with the distance between points or model updates, allowing a nuanced trade-off and preserving cluster structure for group privacy and topological clustering (Galli et al., 2022).
• Homomorphic Encryption and Secure Aggregation: Aggregated updates are computed over encrypted shares, ensuring that the server learns only the combined contribution, not any individual’s raw update (Khan et al., 2024, Cantu-Cervini, 2024).
• Client-side Obfuscation and Personalization Obfuscation: Only partial, aggregated, or encrypted representations (e.g., masked heads or embeddings, aggregated contextual embeddings) are shared. Some variants transmit only model portions (shared base layers), while personal heads remain strictly on-device (Bose et al., 2023, Chen et al., 25 Apr 2025).

4. Optimization, Routing, and Aggregation Pipelines

Optimization in PP-PFL leverages multi-layer aggregation and grouped coordination:

• Personalized Stacking: Clients locally optimize a meta-model atop predictions from all privacy-preserving base models. Held-out datasets are strongly recommended for stacking to avoid overfitting to private models (Cantu-Cervini, 2024).
• Adaptive Local Aggregation: Context embeddings (e.g., irradiance embeddings for PV disaggregation) drive convex combinations of local and global base models, with group similarities dynamically controlling the mixing (Chen et al., 25 Apr 2025).

Clustering or routing operations often rely on divergence measures such as PSI/WPSI, Hellinger, or Jensen-Shannon distances, but recent results indicate that PSI-based grouping provides higher fidelity to the real non-IID structure of federations, leading to empirically superior accuracy and fairness (Jimenez-Gutierrez et al., 23 Dec 2025).

Coordinate-descent, proximal regularization, and meta-learning strategies with stabilization (external regularizers or controlled learning rates) are standard to address non-convexity and heterogeneity (Di et al., 2023, Tran et al., 2023).

5. Empirical Performance, Trade-offs, and Ablation Findings

The empirical landscape of PP-PFL is characterized by:

• Privacy–Utility Degradation: Increasing DP noise (smaller $\varepsilon$) decreases personalization efficacy, with the trade-off typically manifesting as a 1–5% drop in accuracy for privacy budgets near $\varepsilon=1$ (Cooper et al., 30 Jan 2025, Bose et al., 2023, Cantu-Cervini, 2024).
• Scalability and Communication: Communication cost grows linearly with the number of shared models or personalization heads. Selective model-fetching (e.g., top-k relevant bases in stacking) and compression are effective remedies. Clustered aggregation or sub-population clustering mitigate communication and storage overhead by reducing the number of distributed model copies (Jimenez-Gutierrez et al., 23 Dec 2025, Cantu-Cervini, 2024).
• Heterogeneity Handling: Personalization gains are greatest under strong data or label skew; such gains diminish as data become more IID. Clustered or hybrid models (multi-head, stacking) adapt to real group structure more robustly than single global models (Jimenez-Gutierrez et al., 23 Dec 2025, Cantu-Cervini, 2024, Di et al., 2023).
• Personalization Metrics: Accuracy, F1, balanced accuracy, MAE/RMSE (for regression), and client fairness (distance to perfect accuracy) are the dominant evaluation metrics. Contribution and fairness are frequently quantified via variable importance, Shapley values, or mean decrease in impurity (Cantu-Cervini, 2024).
• Comparisons against baselines (FedAvg, FedProx, Clustered FL, FedEM, Ditto, etc.) show that PP-PFL methods consistently achieve higher personalization and global performance under privacy constraints. For example, in (Silva et al., 2022), FedEmbed yields up to 45% improvement in F1 under DP compared to prior PFL approaches; (Jimenez-Gutierrez et al., 23 Dec 2025) reports up to 18% accuracy improvement and 37% relative fairness gain using PSI-driven clustering.

6. Open Challenges, Extensions, and Future Directions

Despite significant progress, several fundamental research challenges persist:

• Scalability in High Dimensions: Metric or Laplace-based DP mechanisms exhibit $O(n)$ noise magnitude in models with millions of parameters, limiting direct applicability to very high-dimensional problems unless further structured or subspace-based perturbation is adopted (Galli et al., 2022).
• Robustness to Client Dropout and Dynamics: Many PP-PFL formulations assume full or synchronous participation; relaxing this assumption to accommodate dynamic, partial participation is an open direction (Di et al., 2023).
• Automatic Structure Discovery: Current clustering- or embedding-based methods often require the number of clusters or sub-populations as a hyperparameter. Bayesian nonparametric approaches or adaptive pruning of canonical models may permit automatic structure synthesis.
• Integration with Cryptographic Primitives: Many empirical PP-PFL systems rely solely on DP; combining cryptographic aggregation, homomorphic encryption, or multi-party computation with personalized learning objectives is a fertile area for research (Khan et al., 2024).
• Extensions to Non-Supervised or Unlabeled Regimes: Most frameworks are formulated for supervised learning; adaptation of PP-PFL to semi-supervised, unsupervised, or contrastive learning remains underexplored (Chen et al., 25 Apr 2025).
• Fairness and Incentive Mechanisms: Attribution via variable-importance or Shapley values provides a foundation for data valuation and reward, but deeper integration of incentive-compatible and fairness-aware objectives is needed (Cantu-Cervini, 2024).
• Theoretical Bounds: Tight privacy–utility trade-off curves and convergence guarantees for deep, non-convex PP-PFL architectures under realistic privacy mechanisms remain a technical gap in most published work (Cooper et al., 30 Jan 2025, Di et al., 2023, Bose et al., 2023, Cantu-Cervini, 2024).

PP-PFL represents a rapidly maturing paradigm that reconciles user-level privacy guarantees with domain-adaptive model performance in federated environments. Innovations in stacking, clustering, metric privacy, meta-learning, and context-adaptive aggregation continue to expand both its practical applicability and theoretical rigor.