Analytic Personalized Federated Learning

• APFL is a personalized federated learning approach that uses a frozen foundation model and dual analytic streams to deliver client-specific models while preserving global knowledge.
• It decomposes the prediction process into a shared primary stream for global generalization and a client-specific refinement stream for local personalization in non-IID settings.
• The method achieves empirical accuracy gains of up to 15.45% while reducing communication rounds via closed-form least-squares solutions instead of iterative gradient updates.

Analytic Personalized Federated Learning (APFL) is a personalized federated learning method introduced in "APFL: Analytic Personalized Federated Learning via Dual-Stream Least Squares" (Fan et al., 14 Aug 2025). It addresses the central personalized federated learning problem of delivering client-specific models while preserving federation-wide knowledge under non-IID data. The method is presented as the first analytic, gradient-free approach to personalized federated learning, and it combines a frozen foundation model with dual-stream least-squares predictors: a shared primary stream for collective generalization and a client-specific refinement stream for local personalization. Its defining claim is a theoretical property termed heterogeneity invariance, under which a client’s personalized model is unaffected by how the data of other clients are partitioned, together with empirical gains of at least $1.10\%-15.45\%$ in accuracy over state-of-the-art baselines (Fan et al., 14 Aug 2025).

1. Problem setting and motivation

Personalized Federated Learning (PFL) aims to provide each client with a tailored model while still leveraging collaborative training across the federation. The main obstacle is client data heterogeneity: when local datasets are non-IID, gradient-based federated updates can degrade collective generalization, and that degradation propagates into downstream personalization. APFL is designed specifically around this failure mode, with the stated goal of addressing the vulnerability of existing PFL methods to non-IID data (Fan et al., 14 Aug 2025).

The method belongs to a broader analytic federated learning line in which global updates are obtained through closed-form least-squares solutions rather than iterative gradient exchange. A relevant precursor is Analytic Federated Learning (AFL), which updates the global model in a single step but suffers when only a single global model must serve heterogeneous clients (Gu et al., 10 Feb 2025). APFL extends the analytic viewpoint from global-only learning to personalized federated learning by decomposing the prediction function into a global component and a client-specific correction (Fan et al., 14 Aug 2025).

This design suggests a shift in emphasis within PFL. Rather than compensating for heterogeneity through more elaborate gradient control, local fine-tuning, or meta-updates, APFL encodes the collaboration and personalization objectives directly as closed-form least-squares problems after feature extraction. A plausible implication is that the method treats heterogeneity less as an optimization instability and more as a modeling problem over shared and residual feature subspaces.

2. Architectural design

APFL is built around two stages: a frozen foundation model and dual-stream analytic models. The foundation model acts as a fixed feature extractor; the paper gives Vision Transformer as an example, and the empirical study uses ViT-MAE-Base for fairness across methods (Fan et al., 14 Aug 2025). Because the backbone is frozen during federated learning, the optimization burden is moved entirely to the analytic heads.

After feature extraction, APFL introduces two separate analytic streams. The primary stream is shared across all clients and is meant to capture what is common across the federation. The refinement stream is client-specific and models local idiosyncrasies. This separation is central: the primary stream handles collective generalization, while the refinement stream personalizes the residual left after the global component is applied (Fan et al., 14 Aug 2025).

The features used by the two streams are not identical. APFL applies separate random projections and non-linear activations to the frozen backbone output, producing distinct analytic feature bases for the primary and refinement streams. For the primary stream, client $k$ computes

$$\mathbf{\Phi}_k = \sigma_P(\mathrm{Backbone}(\mathbf{X}_k)\mathbf{R}_P).$$

For the refinement stream, a separate basis $\mathbf{\Psi}_k$ is extracted after the global stream has been determined. The stated purpose of the random projections and activations is to enhance linear separability for the downstream least-squares solvers (Fan et al., 14 Aug 2025).

The resulting architecture is neither a standard global-model FL pipeline nor a pure local-adaptation scheme. It is a frozen-representation system with analytic linear heads, one shared and one local. In that sense, APFL combines the representational stability associated with pre-trained foundation models and the closed-form solvability of ridge-type least-squares objectives.

3. Objective functions and dual-stream operation

APFL uses mean squared error for analytic tractability and jointly formulates global and personalized learning through two closed-form objectives. Let client $k$ have input $\mathbf{X}_k$, label $\mathbf{Y}_k$, primary feature $\mathbf{\Phi}_k$, and refinement feature $\mathbf{\Psi}_k$. The shared primary stream is

$$\mathbf{\hat G} = \arg\min_{\mathbf{G}} \|\mathbf{Y}_{1:K} - \mathbf{\Phi}_{1:K}\mathbf{G}\|_F^2 + \gamma \|\mathbf{G}\|_F^2,$$

where $k$0 and $k$1 denote concatenated data across all clients. The client-specific refinement stream is

$k$2

At inference, the two components are combined as

$k$3

where $k$4 controls the balance between generalization and personalization (Fan et al., 14 Aug 2025).

The primary stream is computed through client-side sufficient statistics and a server-side analytic aggregation. Each client first solves

$k$5

and uploads its auto-correlation matrix

$k$6

The server then applies a recursive analytic formula based on block-wise matrix inverse to obtain the global primary stream as the exact minimizer of the global least-squares objective: $k$7 A key procedural property is that this aggregation is order-invariant and does not rely on iterative gradient averaging; the server can incorporate client statistics in any order (Fan et al., 14 Aug 2025).

Once $k$8 is returned, each client computes its refinement stream by closed-form local ridge regression: $k$9 This stage explicitly personalizes the residual after subtraction of the global prediction, so the local model is not an independent head trained from scratch; it is a correction to the shared predictor (Fan et al., 14 Aug 2025).

4. Theoretical properties

The theoretical analysis in APFL is organized around four named results. Theorem 1 states that the primary stream $$\mathbf{\Phi}_k = \sigma_P(\mathrm{Backbone}(\mathbf{X}_k)\mathbf{R}_P).$$0 is equivalent to the centralized least-squares minimizer on pooled data. Theorem 2 states that each refinement stream $$\mathbf{\Phi}_k = \sigma_P(\mathrm{Backbone}(\mathbf{X}_k)\mathbf{R}_P).$$1 is the optimal personalized model for the local residual after subtracting the global predictions. Together, these theorems establish closed-form optimality for both components of the architecture (Fan et al., 14 Aug 2025).

The most distinctive claim is Theorem 3, the heterogeneity invariance theorem. For a client $$\mathbf{\Phi}_k = \sigma_P(\mathrm{Backbone}(\mathbf{X}_k)\mathbf{R}_P).$$2, the final personalized model $$\mathbf{\Phi}_k = \sigma_P(\mathrm{Backbone}(\mathbf{X}_k)\mathbf{R}_P).$$3 is stated to be independent of the data distributions of any other client, depending only on the total data across all clients and the local data of client $$\mathbf{\Phi}_k = \sigma_P(\mathrm{Backbone}(\mathbf{X}_k)\mathbf{R}_P).$$4. The paper presents the implication in concrete form: the same pool of examples, if differently partitioned across clients, yields identical personalized models. This is stronger than ordinary robustness to non-IID splits; it is a partition-invariance statement about the analytic solution itself (Fan et al., 14 Aug 2025).

The proof outline attributes this behavior to the fact that the block-wise least-squares solution fully encodes the aggregate data statistics and therefore avoids the bias amplification and client drift associated with gradient-based local model averaging under non-IID data. This suggests that, in APFL, heterogeneity enters through sufficient statistics rather than through iterative optimization trajectories.

Theorem 4 addresses privacy. The uploaded statistics $$\mathbf{\Phi}_k = \sigma_P(\mathrm{Backbone}(\mathbf{X}_k)\mathbf{R}_P).$$5 do not uniquely determine the clients’ local features or labels, so the local data cannot be uniquely reconstructed from the transmitted quantities. The paper also ties the analytic formulation to efficiency: each client needs only a single communication round to share sufficient statistics, rather than the repeated gradient exchanges characteristic of iterative FL (Fan et al., 14 Aug 2025).

5. Empirical behavior, efficiency, and ablations

The empirical study evaluates APFL on CIFAR-100 and ImageNet-R, using non-IID partitions generated by Dirichlet distributions with varied $$\mathbf{\Phi}_k = \sigma_P(\mathrm{Backbone}(\mathbf{X}_k)\mathbf{R}_P).$$6, and considering both 50-client and 100-client settings. Comparisons include classical FL baselines such as FedAvg and FedProx, as well as personalized methods such as Ditto, FedALA, FedSelect, FedPCL, and AFL. All methods use ViT-MAE-Base as the backbone (Fan et al., 14 Aug 2025).

Across these settings, APFL is reported to achieve state-of-the-art test accuracy for every configuration, with margins up to $$\mathbf{\Phi}_k = \sigma_P(\mathrm{Backbone}(\mathbf{X}_k)\mathbf{R}_P).$$7 over the closest strong baseline. Its reported advantage over AFL increases as non-IID severity increases, especially at Dirichlet $$\mathbf{\Phi}_k = \sigma_P(\mathrm{Backbone}(\mathbf{X}_k)\mathbf{R}_P).$$8, which the paper interprets as evidence that the refinement stream is critical. The reported advantage over all baselines also grows with data heterogeneity, while some gradient-based baselines fall below $$\mathbf{\Phi}_k = \sigma_P(\mathrm{Backbone}(\mathbf{X}_k)\mathbf{R}_P).$$9 on ImageNet-R under severe non-IID conditions (Fan et al., 14 Aug 2025).

The systems profile is equally central to the presentation. APFL requires only one communication round and reaches its highest accuracy in that single round, whereas the baselines require more than 200 aggregation rounds for convergence. The paper reports up to $\mathbf{\Psi}_k$0 less communication and computation overhead than iterative baselines, and describes the cost as orders of magnitude lower than gradient-based federated learning (Fan et al., 14 Aug 2025).

Ablation studies further delimit the method’s behavior. The two-stream architecture outperforms a primary-only variant, supporting the claim that both collective generalization and local correction are necessary. The balancing hyperparameter $\mathbf{\Psi}_k$1 exhibits a rising-then-falling effect, which is presented as evidence that neither purely global nor purely local prediction is sufficient. The method is reported to be robust to activation choices and random projection dimensionality provided they are sufficiently large to maintain separability, while regularization through $\mathbf{\Psi}_k$2 and $\mathbf{\Psi}_k$3 is important for avoiding overfitting, with the refinement stream being more sensitive because local data are sparser (Fan et al., 14 Aug 2025).

6. Relation to earlier APFL usage and neighboring personalized FL methods

A recurrent source of confusion is acronym reuse. In earlier personalized federated learning literature, APFL usually referred to "Adaptive Personalized Federated Learning," a 2020 method in which each client forms a personalized predictor as a convex mixture of a local model and a global model,

$\mathbf{\Psi}_k$4

or equivalently as a client-level interpolation between global and local parameters (Deng et al., 2020). That earlier APFL derived a generalization bound for the mixed model, analyzed convergence in strongly convex and nonconvex settings, and adaptively updated the mixing parameter $\mathbf{\Psi}_k$5 during training (Deng et al., 2020). The 2025 Analytic Personalized Federated Learning method uses the same acronym but belongs to a different family: frozen-backbone, gradient-free, dual-stream least-squares learning (Fan et al., 14 Aug 2025).

This distinction matters in comparisons. "Self-Aware Personalized Federated Learning" contrasts its Bayesian uncertainty-driven balancing rule with APFL’s fixed or user-chosen personalization trade-off, arguing that uncertainty quantification should govern initialization, local steps, and aggregation rather than a single mixture coefficient (Chen et al., 2022). "Learn What You Need in Personalized Federated Learning" similarly contrasts per-parameter participation matrices $\mathbf{\Psi}_k$6 with APFL-style scalar client-level participation, positioning Learn2pFed as a finer-grained collaboration mechanism (Lv et al., 2024).

Other empirical studies place the older adaptive APFL as an important baseline but not necessarily a dominant one in every heterogeneity regime. WAFFLE compares against APFL under concept shift and label skew, and reports that WAFFLE and Weight Erosion outperform APFL on those benchmarks while APFL still improves over global-only methods such as FedAvg and SCAFFOLD (Beaussart et al., 2021). A broader empirical survey lists APFL among relevant baselines in the field but does not directly benchmark it in its controlled experiments (Khan et al., 2024).

Within the analytic FL literature, APFL also sits near pFedACnnL, which combines analytic federated learning with gradient-free meta-learning and layer-wise least-squares DNN training (Gu et al., 10 Feb 2025). The two methods share an analytic, gradient-free orientation, but APFL is organized around a frozen foundation model and dual-stream heads, whereas pFedACnnL uses group-wise meta-models and layer-wise analytic adaptation (Gu et al., 10 Feb 2025). This suggests that “analytic personalization” has become a broader design space rather than a single algorithmic template.