---
title: Partial Model Personalization in FL
url: https://www.emergentmind.com/topics/partial-model-personalization
type: topic
---

# Partial Model Personalization in FL

Partial model personalization is a framework in machine learning, especially prevalent in federated learning (FL), where only a selected subset of model parameters are adapted to each user's data, while the remaining parameters are shared among all users. This approach enables adaptation to heterogeneous client distributions and device capabilities with improved resource efficiency compared to full-model personalization, which duplicates all parameters per client. The paradigm is increasingly central to modern FL systems due to its favorable tradeoffs in accuracy, communication, privacy, and system constraints.

## 1. Mathematical Formulation and Core Principles

Partial model personalization decomposes the total parameter vector for client \( k \) as \( \theta_k = [\theta_g, \theta_{p,k}] \in \mathbb{R}^{N_g + N_{p,k}} \), with \( \theta_g \) denoting global/shared parameters and \( \theta_{p,k} \) the client-private personalized parameters. The global parameters are periodically aggregated server-side, while the personalized parameters remain strictly local and are tuned to individual data distributions [2309.01816].

The canonical optimization objective is
\[
\min_{\theta_g, \{\theta_{p,k}\}} \sum_{k=1}^K \frac{D_k}{D} F_k(\theta_g, \theta_{p,k})
\]
where \( F_k \) is the local empirical loss, \( D_k \) the size of client \( k \)'s dataset, and \( D=\sum_k D_k \) [2309.01816, 2204.03809, 2309.17409]. This setup interpolates between (i) classical federated averaging (\( \theta_{p,k}=0 \)), (ii) full personalization (\( \theta_g=0 \)), and (iii) any split in between.

Personalization can be structured at various granularities: layerwise (e.g., only output classifiers, input embeddings, adapters, or self-attention modules), modular (set of specialized adapters/channels), or non-architecturally (e.g., via editing latent representations or token embeddings in LLMs) [2308.09160, 2503.01048].

## 2. Algorithmic Schemes and Optimization Techniques

### Server–Client Alternating Protocols

Partial model personalization is most often realized via a server–client protocol:

- **Weighted model aggregation:** The server disseminates the current shared parameters to clients. Clients perform local updates on both shared (\( \theta_g \)) and personalized (\( \theta_{p,k} \)) parameters via stochastic gradient descent, but at round's end only the updates to \( \theta_g \) are aggregated server-side. \( \theta_{p,k} \) remains private [1912.00818, 2204.03809, 2309.01816].
- **Pruning/adaptation:** Devices may adaptively prune a fraction \( \rho_k \) of shared parameters locally to meet compute or bandwidth constraints, masking out \( \rho_k N_g \) entries of \( \theta_g \) [2309.01816].

Two main local update patterns exist:

| Update Pattern        | Description         | Typical Use Case                                      |
|----------------------|--------------------|-------------------------------------------------------|
| Simultaneous (FedSim)| Update (\(u, v_i\)) jointly  | Simplicity, weak cross-coupling, small models         |
| Alternating (FedAlt) | Sequentially update \(v_i\)—then \(u\)  | Robustness to coupling, strong heterogeneity          |

Alternating updates provide better performance under moderate-to-strong coupling between global and local variables [2204.03809].

### Advanced Optimization: Proximal and ADMM-based Methods

To control client-drift—divergence of client local models from the global shared core—advanced techniques introduce explicit proximal corrections in the optimization:

- **Proximal/penalty regularization:** Quadratic penalties, e.g., \( \frac{\rho}{2} \|u_i-u\|^2 \), stabilize the deviation between client and global shared parameters [2506.04672].
- **Augmented Lagrangian/ADMM:** ADMM-based frameworks such as FedAPM add both first-order (dual multiplier) and second-order corrections to robustly couple local and shared parameters, yielding strong theoretical convergence guarantees under moderate conditions [2506.04672].
- **Decentralized partial personalization:** In fully decentralized networks, partial model training is combined with peer-to-peer consensus (gossip) and local sharpness-aware minimization (SAM) to flatten the shared model landscape across clients without relying on centralized aggregation [2305.15157].

## 3. Specializations in Architectures and Modalities

Partial model personalization is realized across diverse model families and modalities:

- **CNNs and shallow networks:** Personalization is usually applied to the classifier head or shallow adapters, with the global backbone trained across clients [1912.00818].
- **Vision Transformers:** Layerwise sensitivity analysis reveals the classification head and self-attention (SA) layers are most susceptible to distribution skew. FedPerfix achieves high accuracy by partially personalizing only these components using adapter-like prefix modules, while freezing the ViT backbone [2308.09160].
- **Multimodal and missing modality settings:** Hyper-networks conditioned on modality presence, using CLIP embeddings fused with visual summaries, can generate per-sample personalization for late decoder layers, adapting segmentation networks to arbitrary missing modality configurations [2406.01987].
- **Text-to-image diffusion and LLMs:** Both lightweight personalization of small token embeddings (e.g., for quantized U-Nets) via zeroth-order forward passes [2503.14868] and direct representation editing in specific decoder layers (e.g., CHAMELEON [2503.01048]) exemplify partial personalization with minimal fine-tuning.
- **Personalization with consent:** Participatory systems implement partial personalization that allows users to opt-in at test time for group attribute-conditioned models, guaranteeing baseline risk and incentive-compatibility [2302.03874].

## 4. Theoretical Properties and Convergence Analysis

Convergence of partial model personalization algorithms in heterogeneous, non-convex settings is established under standard smoothness, bounded-variance, and (for some methods) bounded gradient dissimilarity assumptions [2309.17409, 2506.04672, 2204.03809]:

- **FedAvg-P/Scaffold-P:** Sharp rates for gradient norm are proven, improving upon prior work and showing that, under full participation, personalized FL reduces to classical FedAvg/SCAFFOLD and matches their best-known rates [2309.17409].
- **FedAPM (ADMM):** By introducing explicit Lagrange multipliers and penalty proximals, global convergence to stationary points is established under broad analytic conditions, with rates ranging from finite to sublinear depending on the Łojasiewicz exponent of the objective [2506.04672].
- **DFedAlt/DFedSalt (decentralized):** In peer-to-peer networks, alternating updates and gossip mixing achieve convergence at a rate \( O(1/\sqrt{T}) \) for the averaged squared gradient norm, with empirical gains in test accuracy and speed [2305.15157].
- **Alternating vs. Simultaneous Local Training:** Alternating updates (FedAlt) are provably superior to simultaneous ones (FedSim) in regimes with moderate batch size and parameter coupling [2204.03809].
- **Pruning-aware convergence:** In resource-constrained FL, model size adaptation and pruning can be jointly optimized alongside bandwidth allocation by leveraging KKT conditions to maximize convergence rate while respecting device-specific latency deadlines [2309.01816].

## 5. Empirical Trends, Memory/Computation Trade-offs, and Guidelines

Extensive benchmarking on image, text, speech, and multimodal datasets consistently demonstrates that partial personalization achieves the majority of the accuracy improvement of full-model personalization, with far lower memory and communication overhead [2204.03809, 2308.09160, 2411.01179]:

| Method        | Personalized Layer(s)       | % Params Personalized | Accuracy Gain   | Comm/Memory Overhead  |
|---------------|----------------------------|----------------------|-----------------|-----------------------|
| Full Finetune | All layers                  | 100                  | Highest (upper bound) | High              |
| Head Only     | Classifier/output           | 1–5                  | ~90% of full         | Low                 |
| Adapter       | Interleaved blocks/modules  | 5–15                 | Matches or > full    | Moderate            |
| FedPerfix     | ViT-SAH+CLS head (prefix)   | ~16                  | SOTA on ViTs        | ~1% extra FLOPs     |
| Hollowed Net  | Partial U-Nets, LoRA        | ~39                  | SOTA, low mem       | ~4 GB vs. 16 GB     |

Empirical guidelines recommend starting with minimal personalization (1–5% of parameters), then gradually adding adapters or expanding the personalized fraction as needed. Communication remains dominated by the global/shared parameters; per-client parameters are never transmitted, enhancing privacy and efficiency [1912.00818, 2204.03809, 2308.09160]. Aggressive model pruning and adaptive allocation strategies can halve system latency in wireless FL [2309.01816].

## 6. Extensions: Personalization in Privacy, Consent, and Special Settings

Partial model personalization supports advanced deployment patterns:

- **Opt-in and consent-driven personalization:** Participatory personalization lets users dynamically opt into reporting sensitive attributes only when it strictly lowers their prediction risk, providing data minimization and fairness guarantees not present in static models or group imputation [2302.03874].
- **Class-missing and few-shot regimes:** In incomplete-class FL, restricted softmax protects classifier proxy stability and the inherited private model mechanism enables long-horizon personalization via momentum aggregation of prior local models [2404.09232].
- **Adaptive reasoning in LLMs:** PersonaDual unifies both objective and persona-aware modes in a single LM via a dual-mode policy, adaptively invoking personalized computation based on task context and achieving near interference-free performance on mixed workloads [2601.08679].

## 7. Limitations, Open Questions, and Future Directions

Despite substantial progress, several limitations and open problems remain:

- **Fraction selection and architecture tuning:** Determining the correct amount and location of personalized parameters is highly task- and data-dependent [1912.00818, 2308.09160].
- **Client drift under partial personalization:** Without proper regularization (e.g., augmented Lagrangian, control variates), divergence between client and server copy of the shared model can degrade generalization [2506.04672].
- **Catastrophic degradation in few-sample users:** Per-user partial models may overfit under severe data scarcity unless additional regularization or diagnostics are introduced [2204.03809].
- **Integration with differential privacy and fairness constraints:** Combining privacy accounting or fairness auditing with partial personalization requires further research [2404.09232].
- **Deployment in fully decentralized networks:** Decentralized partial personalization is theoretically and practically viable, but system-level resilience and convergence under extreme network churn merit continued investigation [2305.15157].

Partial model personalization represents a robust, theoretically-sound approach that balances adaptability and efficiency in heterogeneous, privacy-sensitive, resource-constrained environments, and continues to be an active area of methodological and systems research [2309.01816, 2204.03809, 2309.17409, 2506.04672, 2305.15157, 2503.14868, 2411.01179, 2308.09160, 2503.01048, 1912.00818, 2404.09232, 2406.01987, 2302.03874, 2601.08679].

Source: https://www.emergentmind.com/topics/partial-model-personalization