Partial Model Personalization in FL

Updated 27 April 2026

Partial model personalization is a framework that decomposes model parameters into shared global and client-specific components to tailor models to heterogeneous data.
It employs server–client alternating protocols and techniques like proximal regularization to control client drift and ensure robust convergence in non-convex settings.
Empirical results indicate that partial personalization achieves nearly full-model accuracy gains with significantly lower communication and memory overhead.

Partial model personalization is a framework in machine learning, especially prevalent in federated learning (FL), where only a selected subset of model parameters are adapted to each user's data, while the remaining parameters are shared among all users. This approach enables adaptation to heterogeneous client distributions and device capabilities with improved resource efficiency compared to full-model personalization, which duplicates all parameters per client. The paradigm is increasingly central to modern FL systems due to its favorable tradeoffs in accuracy, communication, privacy, and system constraints.

1. Mathematical Formulation and Core Principles

Partial model personalization decomposes the total parameter vector for client $k$ as $\theta_k = [\theta_g, \theta_{p,k}] \in \mathbb{R}^{N_g + N_{p,k}}$ , with $\theta_g$ denoting global/shared parameters and $\theta_{p,k}$ the client-private personalized parameters. The global parameters are periodically aggregated server-side, while the personalized parameters remain strictly local and are tuned to individual data distributions (Liu et al., 2023).

The canonical optimization objective is

$\min_{\theta_g, \{\theta_{p,k}\}} \sum_{k=1}^K \frac{D_k}{D} F_k(\theta_g, \theta_{p,k})$

where $F_k$ is the local empirical loss, $D_k$ the size of client $k$ 's dataset, and $D=\sum_k D_k$ (Liu et al., 2023, Pillutla et al., 2022, Chen et al., 2023). This setup interpolates between (i) classical federated averaging ( $\theta_{p,k}=0$ ), (ii) full personalization ( $\theta_k = [\theta_g, \theta_{p,k}] \in \mathbb{R}^{N_g + N_{p,k}}$ 0), and (iii) any split in between.

Personalization can be structured at various granularities: layerwise (e.g., only output classifiers, input embeddings, adapters, or self-attention modules), modular (set of specialized adapters/channels), or non-architecturally (e.g., via editing latent representations or token embeddings in LLMs) (Sun et al., 2023, Zhang et al., 2 Mar 2025).

2. Algorithmic Schemes and Optimization Techniques

Server–Client Alternating Protocols

Partial model personalization is most often realized via a server–client protocol:

Weighted model aggregation: The server disseminates the current shared parameters to clients. Clients perform local updates on both shared ( $\theta_k = [\theta_g, \theta_{p,k}] \in \mathbb{R}^{N_g + N_{p,k}}$ 1) and personalized ( $\theta_k = [\theta_g, \theta_{p,k}] \in \mathbb{R}^{N_g + N_{p,k}}$ 2) parameters via stochastic gradient descent, but at round's end only the updates to $\theta_k = [\theta_g, \theta_{p,k}] \in \mathbb{R}^{N_g + N_{p,k}}$ 3 are aggregated server-side. $\theta_k = [\theta_g, \theta_{p,k}] \in \mathbb{R}^{N_g + N_{p,k}}$ 4 remains private (Arivazhagan et al., 2019, Pillutla et al., 2022, Liu et al., 2023).
Pruning/adaptation: Devices may adaptively prune a fraction $\theta_k = [\theta_g, \theta_{p,k}] \in \mathbb{R}^{N_g + N_{p,k}}$ 5 of shared parameters locally to meet compute or bandwidth constraints, masking out $\theta_k = [\theta_g, \theta_{p,k}] \in \mathbb{R}^{N_g + N_{p,k}}$ 6 entries of $\theta_k = [\theta_g, \theta_{p,k}] \in \mathbb{R}^{N_g + N_{p,k}}$ 7 (Liu et al., 2023).

Two main local update patterns exist:

Update Pattern	Description	Typical Use Case
Simultaneous (FedSim)	Update ( $\theta_k = [\theta_g, \theta_{p,k}] \in \mathbb{R}^{N_g + N_{p,k}}$ 8) jointly	Simplicity, weak cross-coupling, small models
Alternating (FedAlt)	Sequentially update $\theta_k = [\theta_g, \theta_{p,k}] \in \mathbb{R}^{N_g + N_{p,k}}$ 9—then $\theta_g$ 0	Robustness to coupling, strong heterogeneity

Alternating updates provide better performance under moderate-to-strong coupling between global and local variables (Pillutla et al., 2022).

Advanced Optimization: Proximal and ADMM-based Methods

To control client-drift—divergence of client local models from the global shared core—advanced techniques introduce explicit proximal corrections in the optimization:

Proximal/penalty regularization: Quadratic penalties, e.g., $\theta_g$ 1, stabilize the deviation between client and global shared parameters (Zhu et al., 5 Jun 2025).
Augmented Lagrangian/ADMM: ADMM-based frameworks such as FedAPM add both first-order (dual multiplier) and second-order corrections to robustly couple local and shared parameters, yielding strong theoretical convergence guarantees under moderate conditions (Zhu et al., 5 Jun 2025).
Decentralized partial personalization: In fully decentralized networks, partial model training is combined with peer-to-peer consensus (gossip) and local sharpness-aware minimization (SAM) to flatten the shared model landscape across clients without relying on centralized aggregation (Shi et al., 2023).

3. Specializations in Architectures and Modalities

Partial model personalization is realized across diverse model families and modalities:

CNNs and shallow networks: Personalization is usually applied to the classifier head or shallow adapters, with the global backbone trained across clients (Arivazhagan et al., 2019).
Vision Transformers: Layerwise sensitivity analysis reveals the classification head and self-attention (SA) layers are most susceptible to distribution skew. FedPerfix achieves high accuracy by partially personalizing only these components using adapter-like prefix modules, while freezing the ViT backbone (Sun et al., 2023).
Multimodal and missing modality settings: Hyper-networks conditioned on modality presence, using CLIP embeddings fused with visual summaries, can generate per-sample personalization for late decoder layers, adapting segmentation networks to arbitrary missing modality configurations (Zhao et al., 2024).
Text-to-image diffusion and LLMs: Both lightweight personalization of small token embeddings (e.g., for quantized U-Nets) via zeroth-order forward passes (Seo et al., 19 Mar 2025) and direct representation editing in specific decoder layers (e.g., CHAMELEON (Zhang et al., 2 Mar 2025)) exemplify partial personalization with minimal fine-tuning.
Personalization with consent: Participatory systems implement partial personalization that allows users to opt-in at test time for group attribute-conditioned models, guaranteeing baseline risk and incentive-compatibility (Joren et al., 2023).

4. Theoretical Properties and Convergence Analysis

Convergence of partial model personalization algorithms in heterogeneous, non-convex settings is established under standard smoothness, bounded-variance, and (for some methods) bounded gradient dissimilarity assumptions (Chen et al., 2023, Zhu et al., 5 Jun 2025, Pillutla et al., 2022):

FedAvg-P/Scaffold-P: Sharp rates for gradient norm are proven, improving upon prior work and showing that, under full participation, personalized FL reduces to classical FedAvg/SCAFFOLD and matches their best-known rates (Chen et al., 2023).
FedAPM (ADMM): By introducing explicit Lagrange multipliers and penalty proximals, global convergence to stationary points is established under broad analytic conditions, with rates ranging from finite to sublinear depending on the Łojasiewicz exponent of the objective (Zhu et al., 5 Jun 2025).
DFedAlt/DFedSalt (decentralized): In peer-to-peer networks, alternating updates and gossip mixing achieve convergence at a rate $\theta_g$ 2 for the averaged squared gradient norm, with empirical gains in test accuracy and speed (Shi et al., 2023).
Alternating vs. Simultaneous Local Training: Alternating updates (FedAlt) are provably superior to simultaneous ones (FedSim) in regimes with moderate batch size and parameter coupling (Pillutla et al., 2022).
Pruning-aware convergence: In resource-constrained FL, model size adaptation and pruning can be jointly optimized alongside bandwidth allocation by leveraging KKT conditions to maximize convergence rate while respecting device-specific latency deadlines (Liu et al., 2023).

5. Empirical Trends, Memory/Computation Trade-offs, and Guidelines

Extensive benchmarking on image, text, speech, and multimodal datasets consistently demonstrates that partial personalization achieves the majority of the accuracy improvement of full-model personalization, with far lower memory and communication overhead (Pillutla et al., 2022, Sun et al., 2023, Cho et al., 2024):

Method	Personalized Layer(s)	% Params Personalized	Accuracy Gain	Comm/Memory Overhead
Full Finetune	All layers	100	Highest (upper bound)	High
Head Only	Classifier/output	1–5	~90% of full	Low
Adapter	Interleaved blocks/modules	5–15	Matches or > full	Moderate
FedPerfix	ViT-SAH+CLS head (prefix)	~16	SOTA on ViTs	~1% extra FLOPs
Hollowed Net	Partial U-Nets, LoRA	~39	SOTA, low mem	~4 GB vs. 16 GB

Empirical guidelines recommend starting with minimal personalization (1–5% of parameters), then gradually adding adapters or expanding the personalized fraction as needed. Communication remains dominated by the global/shared parameters; per-client parameters are never transmitted, enhancing privacy and efficiency (Arivazhagan et al., 2019, Pillutla et al., 2022, Sun et al., 2023). Aggressive model pruning and adaptive allocation strategies can halve system latency in wireless FL (Liu et al., 2023).

Partial model personalization supports advanced deployment patterns:

Opt-in and consent-driven personalization: Participatory personalization lets users dynamically opt into reporting sensitive attributes only when it strictly lowers their prediction risk, providing data minimization and fairness guarantees not present in static models or group imputation (Joren et al., 2023).
Class-missing and few-shot regimes: In incomplete-class FL, restricted softmax protects classifier proxy stability and the inherited private model mechanism enables long-horizon personalization via momentum aggregation of prior local models (Li et al., 2024).
Adaptive reasoning in LLMs: PersonaDual unifies both objective and persona-aware modes in a single LM via a dual-mode policy, adaptively invoking personalized computation based on task context and achieving near interference-free performance on mixed workloads (Liu et al., 13 Jan 2026).

7. Limitations, Open Questions, and Future Directions

Despite substantial progress, several limitations and open problems remain:

Fraction selection and architecture tuning: Determining the correct amount and location of personalized parameters is highly task- and data-dependent (Arivazhagan et al., 2019, Sun et al., 2023).
Client drift under partial personalization: Without proper regularization (e.g., augmented Lagrangian, control variates), divergence between client and server copy of the shared model can degrade generalization (Zhu et al., 5 Jun 2025).
Catastrophic degradation in few-sample users: Per-user partial models may overfit under severe data scarcity unless additional regularization or diagnostics are introduced (Pillutla et al., 2022).
Integration with differential privacy and fairness constraints: Combining privacy accounting or fairness auditing with partial personalization requires further research (Li et al., 2024).
Deployment in fully decentralized networks: Decentralized partial personalization is theoretically and practically viable, but system-level resilience and convergence under extreme network churn merit continued investigation (Shi et al., 2023).