---
title: 'AdaptFED: Adaptive Federated Learning'
url: https://www.emergentmind.com/topics/adaptfed
type: topic
---

# AdaptFED: Adaptive Federated Learning

AdaptFED is a non-canonical term in federated learning literature. In the provided materials, it refers not to one universally standardized algorithm but to several distinct lines of work: adaptive aggregation, locally adaptive client optimization, adapter-based parameter-efficient fine-tuning, adaptive offloading, adaptive aggregation infrastructure, and, in a later transformer paper, an explicit method named AdaptFED [2012.00661] [2307.06306] [2205.10162] [2405.04840] [2107.04271] [2203.12163] [2508.10840]. This suggests that the term is best treated as a literature-level umbrella or alias whose meaning depends on context, with the common denominator being adaptation inside the federated learning pipeline rather than a single canonical protocol.

## 1. Nomenclature and scope

A recurrent misconception is that AdaptFED denotes one fixed federated learning algorithm. The materials instead associate the query with several official method names, and several papers explicitly state that AdaptFED is not the official name they use [2012.00661] [2307.06306]. In some cases the term functions as an alias for a method with a different published name; in others it is used conceptually to group adaptive or adapter-based federated techniques; and in one case it is the actual method name [2508.10840].

| Usage in the materials | Official name | Core mechanism |
|---|---|---|
| Adaptive weighting of client updates | FedAdp | angle-based aggregation weights |
| Locally adaptive federated learning | FedSPS / FedDecSPS | stochastic Polyak stepsizes |
| Adapter-based FedNLP | FedAdapter / GR-T / AdaFL | progressive adapter depth/width |
| Federated recommendation adaptation | FedPA | private and shared low-rank adapters |
| Adaptive offloading for IoT FL | FedAdapt | RL-based layer offloading |
| Adaptive aggregation architecture | AdaFed | serverless scalable aggregation |
| Explicit transformer method | AdaptFED | client-adaptive focal modulation |

The ambiguity is substantive rather than merely terminological. In the optimization papers, adaptation usually means modifying stepsizes, moments, or aggregation weights to cope with heterogeneity [2003.00295] [2212.00974]. In the parameter-efficient fine-tuning papers, adaptation means training only small adapters or low-rank modules while freezing the backbone [2205.10162] [2302.02949] [2505.01075]. In systems papers, adaptation refers to runtime decisions such as offloading and elastic aggregation [2107.04271] [2203.12163].

## 2. Adaptive optimization and aggregation

One major interpretation linked to the query is adaptive federated optimization. In "Adaptive Federated Optimization" [2003.00295], the FedOpt framework separates client-side local training from server-side adaptive optimization. Clients return deltas $\Delta_t^i$, the server aggregates them into $\Delta_t$, forms a pseudo-gradient $g_t := -\Delta_t/\eta_g$, and then applies adaptive server updates such as FedAdagrad, FedAdam, or FedYogi. For example, FedAdagrad uses
$$
v_t = v_{t-1} + g_t \circ g_t,\qquad
w_{t+1} = w_t - \eta_s \frac{g_t}{\sqrt{v_t} + \epsilon},
$$
while FedAdam and FedYogi maintain first- and second-moment states on the server [2003.00295].

"Accelerated Federated Learning with Decoupled Adaptive Optimization" [2207.07223] develops FedDA, which decouples global momentum from local client updates through an ODE-based construction. The method initializes each client with the current global parameter and momentum, evolves a client-local momentum $m^i(t)$, accumulates $P^i=\sum_{t=0}^{T-1} m^i(t+1)$, and then aggregates both $m^i(T)$ and $P^i$ at the server. A one-round full-batch-gradient phase with $T=1$ is then used at the end of training to match centralized optimization dynamics and ensure convergence [2207.07223].

"Faster Adaptive Federated Learning" [2212.00974] pushes this line further with FAFED, a variance-reduced adaptive method for cross-silo FL. Each client forms a local estimator
$$
m_{t,i} := \hat g_{t,i} + (1-\alpha_t)(\bar m_{t-1} - \hat g_{t-1,i}),
$$
maintains a second moment
$$
v_{t,i} := \beta v_{t-1,i} + (1-\beta)(\hat g_{t,i})^2,
$$
and uses a shared preconditioner
$$
A_t := \mathrm{diag}(\sqrt{\bar v_t} + \rho),\qquad
\bar v_t := \frac{1}{N}\sum_i v_{t,i}.
$$
The paper proves that FAFED is the first adaptive FL algorithm to reach the best-known samples $O(\epsilon^{-3})$ and $O(\epsilon^{-2})$ communication rounds to find an $\epsilon$-stationary point without large batches [2212.00974].

Across these works, adaptation is not personalization in the model-architecture sense. It is preconditioning, momentum control, or server-side/state-space design for improving stability, communication efficiency, and convergence under non-IID data [2003.00295] [2207.07223] [2212.00974].

## 3. Client contribution and local geometry

A second cluster of meanings centers on adapting to client contribution or local loss geometry. "Fast-Convergent Federated Learning with Adaptive Weighting" [2012.00661] defines FedAdp, which measures each client’s contribution using the angle between its local gradient and the global gradient. With
$$
g_i^t := \nabla F_i(w^t),\qquad
G^t := \sum_{i\in S_t} \frac{D_i}{\sum_{j\in S_t} D_j} g_i^t,
$$
the method computes
$$
\cos(\theta_i^t) = \frac{\langle g_i^t, G^t\rangle}{\|g_i^t\|\|G^t\|},
\qquad
\theta_i^t = \arccos(\cos(\theta_i^t)),
$$
smooths the angle over rounds, maps it through a decreasing Gompertz-type function, and then uses a softmax to define adaptive aggregation weights $\alpha_i^t$ [2012.00661]. The server update becomes
$$
w^{t+1} = w^t + \sum_{i\in S_t} \alpha_i^t \Delta_i^t.
$$
The paper states that FedAdp achieves a tighter per-round decrease bound than FedAvg’s data-size-only weights, and reports up to 54.1% fewer communication rounds on MNIST and up to 45.4% on FashionMNIST relative to FedAvg [2012.00661].

"Locally Adaptive Federated Learning" [2307.06306] uses fully client-side adaptive stepsizes instead of adaptive aggregation weights. Its principal methods are FedSPS and FedDecSPS. In FedSPS, each client and each local step uses
$$
\eta_i^{(t,k)} =
\min\left\{
\frac{F_i(x_i^{(t,k)};\xi_i^{(t,k)})-\ell_i^*}
{c\|\nabla F_i(x_i^{(t,k)};\xi_i^{(t,k)})\|^2},
\gamma_b
\right\},
$$
with $\ell_i^*=0$ for nonnegative losses in the reported practice [2307.06306]. FedDecSPS replaces the fixed scale with a decreasing schedule $c_t$, which the paper states yields exact convergence with heterogeneity $(\sigma_f>0)$ without resorting to tiny fixed $\gamma_b$ [2307.06306]. The same work reports that FedSPS matches tuned FedAvg in convex settings, outperforms FedAvg and FedAMS on non-convex experiments, and comes with superior generalization performance [2307.06306].

Both methods adapt to heterogeneity, but at different loci. FedAdp changes the aggregation weight assigned to each client’s update [2012.00661]. FedSPS and FedDecSPS change the local optimizer step taken by each client and each minibatch [2307.06306]. The distinction is technically important because one controls inter-client weighting and the other controls intra-client descent geometry.

## 4. Adapter-based and parameter-efficient interpretations

A third, and large, body of material associates AdaptFED with adapter-based parameter-efficient federated learning. In "FedAdapter: Efficient Federated Learning for Modern NLP" [2205.10162], the backbone Transformer is frozen and only adapter modules and the task head are trainable. FedAdapter progressively upgrades adapter depth and width during training and continuously profiles candidate future configurations using trial groups. The reported time-to-accuracy reductions are task-dependent but substantial: on BERT-base at 99% of full-finetune peak accuracy, 20NEWS falls from 44.0 h to 1.3 h, AGNEWS from 31.1 h to 0.2 h, SEMEVAL from 124.3 h to 2.3 h, and ONTONOTES from 76.1 h to 4.5 h; the paper summarizes this as up to 155.5$\times$ faster than vanilla FedNLP and up to 48$\times$ faster than strong baselines [2205.10162].

"Adaptive Parameterization of Deep Learning Models for Federated Learning" [2302.02949] transfers the adapter idea to convolutional vision models through parallel residual adapters. An ImageNet-pretrained ResNet26 is frozen, while each convolutional layer receives a parallel $1\times1$ adapter:
$$
\mathcal{L}_{l,a}(x; W_l, a_l)=g(W_l\ast x + a_l\ast x).
$$
Only the adapters, batch-normalization parameters, and classifier are communicated. For ResNet26, the payload per transmission is reduced from 22.2 MB to 2.58 MB, which the paper describes as roughly 90% and quantifies as approximately 88.4% [2302.02949].

"Federated Adaptation for Foundation Model-based Recommendations" [2405.04840] defines FedPA, in which a frozen recommendation foundation model is augmented with low-rank adapters at each MLP layer, split into user-level private adapters and user-group-level shared adapters, plus a small adaptive gating mechanism. For a linear layer $W_\ell\in\mathbb{R}^{k\times d}$, the adapter adds a low-rank residual $W_a W_b$ with $W_a\in\mathbb{R}^{k\times r}$ and $W_b\in\mathbb{R}^{r\times d}$, so the layer output becomes $y = W_\ell x + W_a W_b x$ [2405.04840]. The paper reports that on KuaiRand, trainable parameters per client drop from 13,649 to 8,189, a 40% reduction, while statistically significant gains are obtained on four benchmark datasets [2405.04840].

"Federated Adapter on Foundation Models: An Out-Of-Distribution Approach" [2505.01075] defines FedOA for OOD generalization in federated foundation models. Each client maintains both a global adapter copy and a personalized adapter, and the personalized adapter is regularized by feature distance to the current global encoder:
$$
\min_{\Phi_e}\; R_e(\Phi_e)+\lambda D(\Phi_e(X^e), \Phi_g^*(X^e)).
$$
The experiments use LoRA adapters on $W_q$ and $W_v$ with rank $r=8$ on a LLaMA-7B backbone initialized via alpaca-LoRA, and report an average leave-one-task-out ROUGE-1 of 59.05 for FedOA, exceeding the listed global and personalized baselines in the paper’s table [2505.01075].

In this literature, adaptation refers primarily to parameter-efficient tuning: the backbone remains frozen, the communicated state is small, and heterogeneity is handled through adapter depth, width, low-rank factors, private/shared branches, or feature-space regularization rather than full-model fine-tuning [2205.10162] [2302.02949] [2405.04840] [2505.01075].

## 5. Systems-level meanings: offloading and elastic aggregation

Some uses associated with the query are systems-oriented rather than optimization-oriented. "FedAdapt: Adaptive Offloading for IoT Devices in Federated Learning" [2107.04271] addresses constrained compute, stragglers, and changing bandwidth by learning where to split a DNN between device and server. If the cut is after layer $k$, a single local iteration is modeled as
$$
T_{\mathrm{iter}}(k)=\sum_{i=1}^{k} t_i^{\mathrm{dev}}(b)
+\sum_{i=k+1}^{L} t_i^{\mathrm{srv}}(b)
+C_{\mathrm{up}}(k)+C_{\mathrm{down}}(k)+q^{\mathrm{srv}},
$$
and the framework uses clustering plus PPO to choose offloading actions [2107.04271]. The paper reports that offloading reduces the training time of a typical IoT device by over half compared to classic FL, reduces the training time of extreme stragglers and the overall training time by up to 57%, and under changing bandwidth reduces training time by up to 40% without sacrificing accuracy [2107.04271].

"Adaptive Aggregation For Federated Learning" [2203.12163] uses AdaFed to denote a scalable aggregation architecture built from serverless/cloud functions, Ray, and Kafka rather than a new model update rule. Updates and partial aggregates flow through Kafka topics, Ray triggers short-lived aggregation functions, and the system elastically scales to participant joins and leaves [2203.12163]. The paper states that its prototype scales to thousands of participants and achieves a >90% reduction in resource requirements and cost with minimal impact on aggregation latency [2203.12163].

These works broaden the meaning of adaptation in FL. Here adaptation is neither per-coordinate optimization nor parameter-efficient fine-tuning. It is runtime control over execution placement or aggregation infrastructure, motivated by systems heterogeneity, bandwidth variation, and large-scale participation [2107.04271] [2203.12163].

## 6. AdaptFED as an explicit method name

The most literal use of the term appears in "Generalizable Federated Learning using Client Adaptive Focal Modulation" [2508.10840], where AdaptFED is the official method name and an explicit extension of TransFed. The method uses a focal-modulation transformer backbone, a server-side hypernetwork, and task-aware client embeddings to generate client-specific projection parameters. For client $i$,
$$
P_i = h(\phi; z_i),\qquad
P_{i,\tau} = h(\phi; [z_i,\tau]),
$$
where $\phi$ are hypernetwork parameters, $z_i$ is a client embedding, and $\tau$ is an optional task descriptor [2508.10840]. Shared backbone parameters $\xi$ are aggregated globally, while focal-modulation parameters are personalized through the hypernetwork.

The global objective is written as
$$
\min_{\xi,\phi,\{z_i\}}
\sum_{i=1}^N \frac{m_i}{S} K_i(h(\phi; z_i), \xi)
+\mathcal{R}(\xi,\phi,\{z_i\}),
$$
with local training performed on $\theta_i=(\xi_i,P_i)$ and server-side hypernetwork updates obtained by chain rule through the observed modulation updates $\Delta P_i$ [2508.10840]. The paper also introduces a communication-efficient low-rank conditioning variant,
$$
P_{i,\tau}^{\ell} \approx U_\ell \,\mathrm{diag}(\gamma_{i,\tau}^{\ell})\, V_\ell^\top,
$$
with $r_\ell \ll \min(d_\ell,k_\ell)$, so that the hypernetwork outputs only $\gamma_{i,\tau}^{\ell}$ rather than a full projection matrix [2508.10840].

The theoretical contribution is a generalization bound for hypernetwork-driven personalization under boundedness and Lipschitz assumptions. The bound contains explicit dependence on sample size $M$, number of clients $N$, hypothesis complexity $d$, and Lipschitz constants $L_h$, $L_\phi$, $L_z$, and $L_\xi$, and the paper interprets the shared hypernetwork as reducing hypothesis complexity relative to per-client parameterization [2508.10840]. Empirically, the method is evaluated on RSNA, Kermany, CIFAR-10, CIFAR-100, Office-Caltech, and Shakespeare. Reported results include 92.67 ± 0.74% on RSNA with 100 pathological clients, 93.47 ± 0.75% on CIFAR-10 with 50 pathological clients, 71.96 ± 0.54% on CIFAR-100 with 50 pathological clients, 96.9% average on Office-Caltech in the Fed-SFDA setting, and 87.57 ± 0.47% on Shakespeare [2508.10840].

Within the broader literature, this explicit AdaptFED method is distinctive because it combines three elements that are otherwise distributed across separate subfields: personalization through client-conditioned generation, generalization through a shared transformer backbone, and communication reduction through low-rank hypernetwork conditioning [2508.10840]. In that narrower sense, it is the clearest candidate for a canonical method actually named AdaptFED, even though the surrounding literature uses the same query term for several other adaptive and adapter-based federated paradigms.

Source: https://www.emergentmind.com/topics/adaptfed