AdaptFED: Adaptive Federated Learning

• AdaptFED is a term that encompasses multiple adaptive strategies within federated learning pipelines, highlighting context-dependent adaptations rather than a single algorithm.
• It covers methods such as adaptive optimization, locally adaptive client updates, and adapter-based parameter-efficient tuning to enhance convergence and reduce communication rounds.
• AdaptFED also includes system-level approaches like adaptive offloading and scalable aggregation that boost efficiency in resource-constrained and large-scale environments.

AdaptFED is a non-canonical term in federated learning literature. In the provided materials, it refers not to one universally standardized algorithm but to several distinct lines of work: adaptive aggregation, locally adaptive client optimization, adapter-based parameter-efficient fine-tuning, adaptive offloading, adaptive aggregation infrastructure, and, in a later transformer paper, an explicit method named AdaptFED (Wu et al., 2020, Mukherjee et al., 2023, Cai et al., 2022, Zhang et al., 2024, Wu et al., 2021, Jayaram et al., 2022, Ashraf et al., 14 Aug 2025). This suggests that the term is best treated as a literature-level umbrella or alias whose meaning depends on context, with the common denominator being adaptation inside the federated learning pipeline rather than a single canonical protocol.

1. Nomenclature and scope

A recurrent misconception is that AdaptFED denotes one fixed federated learning algorithm. The materials instead associate the query with several official method names, and several papers explicitly state that AdaptFED is not the official name they use (Wu et al., 2020, Mukherjee et al., 2023). In some cases the term functions as an alias for a method with a different published name; in others it is used conceptually to group adaptive or adapter-based federated techniques; and in one case it is the actual method name (Ashraf et al., 14 Aug 2025).

Usage in the materials	Official name	Core mechanism
Adaptive weighting of client updates	FedAdp	angle-based aggregation weights
Locally adaptive federated learning	FedSPS / FedDecSPS	stochastic Polyak stepsizes
Adapter-based FedNLP	FedAdapter / GR-T / AdaFL	progressive adapter depth/width
Federated recommendation adaptation	FedPA	private and shared low-rank adapters
Adaptive offloading for IoT FL	FedAdapt	RL-based layer offloading
Adaptive aggregation architecture	AdaFed	serverless scalable aggregation
Explicit transformer method	AdaptFED	client-adaptive focal modulation

The ambiguity is substantive rather than merely terminological. In the optimization papers, adaptation usually means modifying stepsizes, moments, or aggregation weights to cope with heterogeneity (Reddi et al., 2020, Wu et al., 2022). In the parameter-efficient fine-tuning papers, adaptation means training only small adapters or low-rank modules while freezing the backbone (Cai et al., 2022, Elvebakken et al., 2023, Yang et al., 2 May 2025). In systems papers, adaptation refers to runtime decisions such as offloading and elastic aggregation (Wu et al., 2021, Jayaram et al., 2022).

2. Adaptive optimization and aggregation

One major interpretation linked to the query is adaptive federated optimization. In "Adaptive Federated Optimization" (Reddi et al., 2020), the FedOpt framework separates client-side local training from server-side adaptive optimization. Clients return deltas $\Delta_t^i$, the server aggregates them into $\Delta_t$, forms a pseudo-gradient $g_t := -\Delta_t/\eta_g$, and then applies adaptive server updates such as FedAdagrad, FedAdam, or FedYogi. For example, FedAdagrad uses

$$v_t = v_{t-1} + g_t \circ g_t,\qquad w_{t+1} = w_t - \eta_s \frac{g_t}{\sqrt{v_t} + \epsilon},$$

while FedAdam and FedYogi maintain first- and second-moment states on the server (Reddi et al., 2020).

"Accelerated Federated Learning with Decoupled Adaptive Optimization" (Jin et al., 2022) develops FedDA, which decouples global momentum from local client updates through an ODE-based construction. The method initializes each client with the current global parameter and momentum, evolves a client-local momentum $m^i(t)$, accumulates $P^i=\sum_{t=0}^{T-1} m^i(t+1)$, and then aggregates both $m^i(T)$ and $P^i$ at the server. A one-round full-batch-gradient phase with $T=1$ is then used at the end of training to match centralized optimization dynamics and ensure convergence (Jin et al., 2022).

"Faster Adaptive Federated Learning" (Wu et al., 2022) pushes this line further with FAFED, a variance-reduced adaptive method for cross-silo FL. Each client forms a local estimator

$$m_{t,i} := \hat g_{t,i} + (1-\alpha_t)(\bar m_{t-1} - \hat g_{t-1,i}),$$

maintains a second moment

$\Delta_t$0

and uses a shared preconditioner

$\Delta_t$1

The paper proves that FAFED is the first adaptive FL algorithm to reach the best-known samples $\Delta_t$2 and $\Delta_t$3 communication rounds to find an $\Delta_t$4-stationary point without large batches (Wu et al., 2022).

Across these works, adaptation is not personalization in the model-architecture sense. It is preconditioning, momentum control, or server-side/state-space design for improving stability, communication efficiency, and convergence under non-IID data (Reddi et al., 2020, Jin et al., 2022, Wu et al., 2022).

3. Client contribution and local geometry

A second cluster of meanings centers on adapting to client contribution or local loss geometry. "Fast-Convergent Federated Learning with Adaptive Weighting" (Wu et al., 2020) defines FedAdp, which measures each client’s contribution using the angle between its local gradient and the global gradient. With

$\Delta_t$5

the method computes

$\Delta_t$6

smooths the angle over rounds, maps it through a decreasing Gompertz-type function, and then uses a softmax to define adaptive aggregation weights $\Delta_t$7 (Wu et al., 2020). The server update becomes

$\Delta_t$8

The paper states that FedAdp achieves a tighter per-round decrease bound than FedAvg’s data-size-only weights, and reports up to 54.1% fewer communication rounds on MNIST and up to 45.4% on FashionMNIST relative to FedAvg (Wu et al., 2020).

"Locally Adaptive Federated Learning" (Mukherjee et al., 2023) uses fully client-side adaptive stepsizes instead of adaptive aggregation weights. Its principal methods are FedSPS and FedDecSPS. In FedSPS, each client and each local step uses

$\Delta_t$9

with $g_t := -\Delta_t/\eta_g$0 for nonnegative losses in the reported practice (Mukherjee et al., 2023). FedDecSPS replaces the fixed scale with a decreasing schedule $g_t := -\Delta_t/\eta_g$1, which the paper states yields exact convergence with heterogeneity $g_t := -\Delta_t/\eta_g$2 without resorting to tiny fixed $g_t := -\Delta_t/\eta_g$3 (Mukherjee et al., 2023). The same work reports that FedSPS matches tuned FedAvg in convex settings, outperforms FedAvg and FedAMS on non-convex experiments, and comes with superior generalization performance (Mukherjee et al., 2023).

Both methods adapt to heterogeneity, but at different loci. FedAdp changes the aggregation weight assigned to each client’s update (Wu et al., 2020). FedSPS and FedDecSPS change the local optimizer step taken by each client and each minibatch (Mukherjee et al., 2023). The distinction is technically important because one controls inter-client weighting and the other controls intra-client descent geometry.

4. Adapter-based and parameter-efficient interpretations

A third, and large, body of material associates AdaptFED with adapter-based parameter-efficient federated learning. In "FedAdapter: Efficient Federated Learning for Modern NLP" (Cai et al., 2022), the backbone Transformer is frozen and only adapter modules and the task head are trainable. FedAdapter progressively upgrades adapter depth and width during training and continuously profiles candidate future configurations using trial groups. The reported time-to-accuracy reductions are task-dependent but substantial: on BERT-base at 99% of full-finetune peak accuracy, 20NEWS falls from 44.0 h to 1.3 h, AGNEWS from 31.1 h to 0.2 h, SEMEVAL from 124.3 h to 2.3 h, and ONTONOTES from 76.1 h to 4.5 h; the paper summarizes this as up to 155.5$g_t := -\Delta_t/\eta_g$4 faster than vanilla FedNLP and up to 48$g_t := -\Delta_t/\eta_g$5 faster than strong baselines (Cai et al., 2022).

"Adaptive Parameterization of Deep Learning Models for Federated Learning" (Elvebakken et al., 2023) transfers the adapter idea to convolutional vision models through parallel residual adapters. An ImageNet-pretrained ResNet26 is frozen, while each convolutional layer receives a parallel $g_t := -\Delta_t/\eta_g$6 adapter:

$g_t := -\Delta_t/\eta_g$7

Only the adapters, batch-normalization parameters, and classifier are communicated. For ResNet26, the payload per transmission is reduced from 22.2 MB to 2.58 MB, which the paper describes as roughly 90% and quantifies as approximately 88.4% (Elvebakken et al., 2023).

"Federated Adaptation for Foundation Model-based Recommendations" (Zhang et al., 2024) defines FedPA, in which a frozen recommendation foundation model is augmented with low-rank adapters at each MLP layer, split into user-level private adapters and user-group-level shared adapters, plus a small adaptive gating mechanism. For a linear layer $g_t := -\Delta_t/\eta_g$8, the adapter adds a low-rank residual $g_t := -\Delta_t/\eta_g$9 with $$v_t = v_{t-1} + g_t \circ g_t,\qquad w_{t+1} = w_t - \eta_s \frac{g_t}{\sqrt{v_t} + \epsilon},$$0 and $$v_t = v_{t-1} + g_t \circ g_t,\qquad w_{t+1} = w_t - \eta_s \frac{g_t}{\sqrt{v_t} + \epsilon},$$1, so the layer output becomes $$v_t = v_{t-1} + g_t \circ g_t,\qquad w_{t+1} = w_t - \eta_s \frac{g_t}{\sqrt{v_t} + \epsilon},$$2 (Zhang et al., 2024). The paper reports that on KuaiRand, trainable parameters per client drop from 13,649 to 8,189, a 40% reduction, while statistically significant gains are obtained on four benchmark datasets (Zhang et al., 2024).

"Federated Adapter on Foundation Models: An Out-Of-Distribution Approach" (Yang et al., 2 May 2025) defines FedOA for OOD generalization in federated foundation models. Each client maintains both a global adapter copy and a personalized adapter, and the personalized adapter is regularized by feature distance to the current global encoder:

$$v_t = v_{t-1} + g_t \circ g_t,\qquad w_{t+1} = w_t - \eta_s \frac{g_t}{\sqrt{v_t} + \epsilon},$$3

The experiments use LoRA adapters on $$v_t = v_{t-1} + g_t \circ g_t,\qquad w_{t+1} = w_t - \eta_s \frac{g_t}{\sqrt{v_t} + \epsilon},$$4 and $$v_t = v_{t-1} + g_t \circ g_t,\qquad w_{t+1} = w_t - \eta_s \frac{g_t}{\sqrt{v_t} + \epsilon},$$5 with rank $$v_t = v_{t-1} + g_t \circ g_t,\qquad w_{t+1} = w_t - \eta_s \frac{g_t}{\sqrt{v_t} + \epsilon},$$6 on a LLaMA-7B backbone initialized via alpaca-LoRA, and report an average leave-one-task-out ROUGE-1 of 59.05 for FedOA, exceeding the listed global and personalized baselines in the paper’s table (Yang et al., 2 May 2025).

In this literature, adaptation refers primarily to parameter-efficient tuning: the backbone remains frozen, the communicated state is small, and heterogeneity is handled through adapter depth, width, low-rank factors, private/shared branches, or feature-space regularization rather than full-model fine-tuning (Cai et al., 2022, Elvebakken et al., 2023, Zhang et al., 2024, Yang et al., 2 May 2025).

5. Systems-level meanings: offloading and elastic aggregation

Some uses associated with the query are systems-oriented rather than optimization-oriented. "FedAdapt: Adaptive Offloading for IoT Devices in Federated Learning" (Wu et al., 2021) addresses constrained compute, stragglers, and changing bandwidth by learning where to split a DNN between device and server. If the cut is after layer $$v_t = v_{t-1} + g_t \circ g_t,\qquad w_{t+1} = w_t - \eta_s \frac{g_t}{\sqrt{v_t} + \epsilon},$$7, a single local iteration is modeled as

$$v_t = v_{t-1} + g_t \circ g_t,\qquad w_{t+1} = w_t - \eta_s \frac{g_t}{\sqrt{v_t} + \epsilon},$$8

and the framework uses clustering plus PPO to choose offloading actions (Wu et al., 2021). The paper reports that offloading reduces the training time of a typical IoT device by over half compared to classic FL, reduces the training time of extreme stragglers and the overall training time by up to 57%, and under changing bandwidth reduces training time by up to 40% without sacrificing accuracy (Wu et al., 2021).

"Adaptive Aggregation For Federated Learning" (Jayaram et al., 2022) uses AdaFed to denote a scalable aggregation architecture built from serverless/cloud functions, Ray, and Kafka rather than a new model update rule. Updates and partial aggregates flow through Kafka topics, Ray triggers short-lived aggregation functions, and the system elastically scales to participant joins and leaves (Jayaram et al., 2022). The paper states that its prototype scales to thousands of participants and achieves a >90% reduction in resource requirements and cost with minimal impact on aggregation latency (Jayaram et al., 2022).

These works broaden the meaning of adaptation in FL. Here adaptation is neither per-coordinate optimization nor parameter-efficient fine-tuning. It is runtime control over execution placement or aggregation infrastructure, motivated by systems heterogeneity, bandwidth variation, and large-scale participation (Wu et al., 2021, Jayaram et al., 2022).

6. AdaptFED as an explicit method name

The most literal use of the term appears in "Generalizable Federated Learning using Client Adaptive Focal Modulation" (Ashraf et al., 14 Aug 2025), where AdaptFED is the official method name and an explicit extension of TransFed. The method uses a focal-modulation transformer backbone, a server-side hypernetwork, and task-aware client embeddings to generate client-specific projection parameters. For client $$v_t = v_{t-1} + g_t \circ g_t,\qquad w_{t+1} = w_t - \eta_s \frac{g_t}{\sqrt{v_t} + \epsilon},$$9,

$m^i(t)$0

where $m^i(t)$1 are hypernetwork parameters, $m^i(t)$2 is a client embedding, and $m^i(t)$3 is an optional task descriptor (Ashraf et al., 14 Aug 2025). Shared backbone parameters $m^i(t)$4 are aggregated globally, while focal-modulation parameters are personalized through the hypernetwork.

The global objective is written as

$m^i(t)$5

with local training performed on $m^i(t)$6 and server-side hypernetwork updates obtained by chain rule through the observed modulation updates $m^i(t)$7 (Ashraf et al., 14 Aug 2025). The paper also introduces a communication-efficient low-rank conditioning variant,

$m^i(t)$8

with $m^i(t)$9, so that the hypernetwork outputs only $P^i=\sum_{t=0}^{T-1} m^i(t+1)$0 rather than a full projection matrix (Ashraf et al., 14 Aug 2025).

The theoretical contribution is a generalization bound for hypernetwork-driven personalization under boundedness and Lipschitz assumptions. The bound contains explicit dependence on sample size $P^i=\sum_{t=0}^{T-1} m^i(t+1)$1, number of clients $P^i=\sum_{t=0}^{T-1} m^i(t+1)$2, hypothesis complexity $P^i=\sum_{t=0}^{T-1} m^i(t+1)$3, and Lipschitz constants $P^i=\sum_{t=0}^{T-1} m^i(t+1)$4, $P^i=\sum_{t=0}^{T-1} m^i(t+1)$5, $P^i=\sum_{t=0}^{T-1} m^i(t+1)$6, and $P^i=\sum_{t=0}^{T-1} m^i(t+1)$7, and the paper interprets the shared hypernetwork as reducing hypothesis complexity relative to per-client parameterization (Ashraf et al., 14 Aug 2025). Empirically, the method is evaluated on RSNA, Kermany, CIFAR-10, CIFAR-100, Office-Caltech, and Shakespeare. Reported results include 92.67 ± 0.74% on RSNA with 100 pathological clients, 93.47 ± 0.75% on CIFAR-10 with 50 pathological clients, 71.96 ± 0.54% on CIFAR-100 with 50 pathological clients, 96.9% average on Office-Caltech in the Fed-SFDA setting, and 87.57 ± 0.47% on Shakespeare (Ashraf et al., 14 Aug 2025).

Within the broader literature, this explicit AdaptFED method is distinctive because it combines three elements that are otherwise distributed across separate subfields: personalization through client-conditioned generation, generalization through a shared transformer backbone, and communication reduction through low-rank hypernetwork conditioning (Ashraf et al., 14 Aug 2025). In that narrower sense, it is the clearest candidate for a canonical method actually named AdaptFED, even though the surrounding literature uses the same query term for several other adaptive and adapter-based federated paradigms.