FedBCGD+: Federated Block Coordinate Descent

• FedBCGD+ is a family of federated learning methods that applies block-coordinate strategies, with one variant optimizing coupled global and personalized models and the other enhancing communication efficiency.
• The 2020 variant employs a quadratic penalty and hierarchical, asynchronous cloud-edge updates to balance personalization with global consensus and reduce latency.
• The 2026 variant partitions model parameters into blocks and incorporates client-drift control with SVRG corrections, significantly cutting communication costs while accelerating convergence.

Searching arXiv for the cited FedBCGD/FedBCGD+ papers and closely related terminology. arXiv search query: "FedBCGD+ federated block coordinate descent" FedBCGD+ is a name used in arXiv federated-learning literature for two distinct block-coordinate methods. In "Federated Block Coordinate Descent Scheme for Learning Global and Personalized Models" (Wu et al., 2020), FedBCGD⁺ denotes a federated block-coordinate-descent algorithm for joint learning of a global model and personalized local models under a quadratic penalty, together with a hierarchical cloud-to-edge and intra-cloud communication structure. In "FedBCD: Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learning" (Liu et al., 5 Mar 2026), FedBCGD+ denotes an accelerated communication-efficient method that partitions parameters into blocks, transmits only selected blocks, and augments client updates with client-drift control and stochastic variance reduction. The shared label reflects a common block-coordinate perspective, but the optimization objectives, communication models, and theoretical emphases differ substantially.

1. Terminology and problem setting

In the 2020 formulation, the central objective is personalization under federated privacy constraints. The method minimizes a coupled objective over a global model $z \in \mathbb{R}^d$ and client-specific models $w_1,\dots,w_N \in \mathbb{R}^d$:

$$\min_{z \in \mathbb{R}^d,\;\{w_i\}\subset \mathbb{R}^d} F(z,\{w_i\}) \;\triangleq\; \sum_{i=1}^N f_i(w_i) + \frac{\mu}{2}\sum_{i=1}^N \|w_i-z\|^2.$$

Here $f_i(w_i)=\mathbb{E}_{\xi\sim D_i}[\ell(w_i;\xi)]$ is the expected loss on device $i$'s local distribution $D_i$, $z$ is the global consensus model, $w_i$ is device $i$'s personalized model, and $\mu>0$ is a regularization hyperparameter often called the personalization weight (Wu et al., 2020).

In the 2026 formulation, the central objective is communication efficiency for large-scale models such as Vision Transformer. The method splits model parameters into several blocks, including a shared block, and enables uploading a specific parameter block by each client. FedBCGD+ extends the basic FedBCGD framework with a client-drift control variate $w_1,\dots,w_N \in \mathbb{R}^d$0 and an SVRG-style variance-reduction correction (Liu et al., 5 Mar 2026).

A plausible implication is that the term "FedBCGD+" should not be treated as a single canonical algorithm without reference to its source paper. In the 2020 line, "plus" refers to momentum on the global model and an adaptive penalty schedule; in the 2026 line, "plus" refers to an accelerated block-coordinate scheme with client drift control and stochastic variance reduction (Wu et al., 2020, Liu et al., 5 Mar 2026).

2. Personalized FedBCGD⁺: joint global and local models

The 2020 method begins from a quadratic-penalty formulation that couples a global template to personalized client models. The penalty term $w_1,\dots,w_N \in \mathbb{R}^d$1 tethers each $w_1,\dots,w_N \in \mathbb{R}^d$2 to $w_1,\dots,w_N \in \mathbb{R}^d$3, so that learning is neither purely global nor fully decoupled (Wu et al., 2020). The paper states that by driving $w_1,\dots,w_N \in \mathbb{R}^d$4 one recovers pure personalization $w_1,\dots,w_N \in \mathbb{R}^d$5, whereas $w_1,\dots,w_N \in \mathbb{R}^d$6 decouples the clients and each solves its own problem. In practice, $w_1,\dots,w_N \in \mathbb{R}^d$7 is chosen to trade off global generalization against local tailoring (Wu et al., 2020).

The synchronous algorithm is organized as a two-step block-coordinate procedure. At communication round $w_1,\dots,w_N \in \mathbb{R}^d$8, the cloud broadcasts the current global model $w_1,\dots,w_N \in \mathbb{R}^d$9 to a subset $$\min_{z \in \mathbb{R}^d,\;\{w_i\}\subset \mathbb{R}^d} F(z,\{w_i\}) \;\triangleq\; \sum_{i=1}^N f_i(w_i) + \frac{\mu}{2}\sum_{i=1}^N \|w_i-z\|^2.$$0 of participating clients. Each client $$\min_{z \in \mathbb{R}^d,\;\{w_i\}\subset \mathbb{R}^d} F(z,\{w_i\}) \;\triangleq\; \sum_{i=1}^N f_i(w_i) + \frac{\mu}{2}\sum_{i=1}^N \|w_i-z\|^2.$$1 updates its local block while holding $$\min_{z \in \mathbb{R}^d,\;\{w_i\}\subset \mathbb{R}^d} F(z,\{w_i\}) \;\triangleq\; \sum_{i=1}^N f_i(w_i) + \frac{\mu}{2}\sum_{i=1}^N \|w_i-z\|^2.$$2 fixed:

$$\min_{z \in \mathbb{R}^d,\;\{w_i\}\subset \mathbb{R}^d} F(z,\{w_i\}) \;\triangleq\; \sum_{i=1}^N f_i(w_i) + \frac{\mu}{2}\sum_{i=1}^N \|w_i-z\|^2.$$3

The implementation uses a single gradient step,

$$\min_{z \in \mathbb{R}^d,\;\{w_i\}\subset \mathbb{R}^d} F(z,\{w_i\}) \;\triangleq\; \sum_{i=1}^N f_i(w_i) + \frac{\mu}{2}\sum_{i=1}^N \|w_i-z\|^2.$$4

with step size $$\min_{z \in \mathbb{R}^d,\;\{w_i\}\subset \mathbb{R}^d} F(z,\{w_i\}) \;\triangleq\; \sum_{i=1}^N f_i(w_i) + \frac{\mu}{2}\sum_{i=1}^N \|w_i-z\|^2.$$5 (Wu et al., 2020).

The cloud then updates the global block by solving

$$\min_{z \in \mathbb{R}^d,\;\{w_i\}\subset \mathbb{R}^d} F(z,\{w_i\}) \;\triangleq\; \sum_{i=1}^N f_i(w_i) + \frac{\mu}{2}\sum_{i=1}^N \|w_i-z\|^2.$$6

which yields the gradient-style update

$$\min_{z \in \mathbb{R}^d,\;\{w_i\}\subset \mathbb{R}^d} F(z,\{w_i\}) \;\triangleq\; \sum_{i=1}^N f_i(w_i) + \frac{\mu}{2}\sum_{i=1}^N \|w_i-z\|^2.$$7

where $$\min_{z \in \mathbb{R}^d,\;\{w_i\}\subset \mathbb{R}^d} F(z,\{w_i\}) \;\triangleq\; \sum_{i=1}^N f_i(w_i) + \frac{\mu}{2}\sum_{i=1}^N \|w_i-z\|^2.$$8 (Wu et al., 2020). For the special choice $$\min_{z \in \mathbb{R}^d,\;\{w_i\}\subset \mathbb{R}^d} F(z,\{w_i\}) \;\triangleq\; \sum_{i=1}^N f_i(w_i) + \frac{\mu}{2}\sum_{i=1}^N \|w_i-z\|^2.$$9, the update reduces to the model-averaging rule

$f_i(w_i)=\mathbb{E}_{\xi\sim D_i}[\ell(w_i;\xi)]$0

This formulation makes block structure explicit at the level of optimization variables rather than parameter partitions. One block is the shared global template $f_i(w_i)=\mathbb{E}_{\xi\sim D_i}[\ell(w_i;\xi)]$1, and the other blocks are the personalized client models $f_i(w_i)=\mathbb{E}_{\xi\sim D_i}[\ell(w_i;\xi)]$2. This suggests that the method is aimed at statistically heterogeneous data settings in which a single fully shared model is not adequate.

3. Hierarchical communication and asynchronous cloud operation

A defining feature of the 2020 FedBCGD⁺ line is its two-layer communication structure: cloud-to-edge and intra-cloud among sub-cloud servers (Wu et al., 2020). In large-scale deployment, the cloud may itself be geographically distributed. The paper arranges cloud servers in a fast connectivity graph $f_i(w_i)=\mathbb{E}_{\xi\sim D_i}[\ell(w_i;\xi)]$3 and runs a lightweight consensus (gossip) or push-sum protocol.

The asynchronous procedure is described concretely. Whenever one server receives an updated $f_i(w_i)=\mathbb{E}_{\xi\sim D_i}[\ell(w_i;\xi)]$4 from edge $f_i(w_i)=\mathbb{E}_{\xi\sim D_i}[\ell(w_i;\xi)]$5, it incorporates it into its local sum, asynchronously exchanges partial sums with its neighbors in $f_i(w_i)=\mathbb{E}_{\xi\sim D_i}[\ell(w_i;\xi)]$6, and applies a local $f_i(w_i)=\mathbb{E}_{\xi\sim D_i}[\ell(w_i;\xi)]$7-update step of the same form as the synchronous global update but with the partially aggregated average (Wu et al., 2020). Under mild conditions—connected $f_i(w_i)=\mathbb{E}_{\xi\sim D_i}[\ell(w_i;\xi)]$8 and proper consensus weights—the stale aggregates converge to the true average exponentially fast in the intra-cloud iteration count.

The paper explicitly frames this asynchronous protocol as being inspired by multi-agent consensus technique and states that it has the potential for large gains in latency compared to a synchronous setting when the edge-device updates are intermittent (Wu et al., 2020). The significance of this design is not merely reduced idle time at slow clients. It also moves part of the aggregation burden into a distributed cloud layer, which is distinct from standard single-server federated averaging.

This architecture should not be conflated with the 2026 block-communication method. In the latter, block structure refers to partitioned parameter coordinates, and the server aggregates per-parameter-block updates in parallel. The 2026 paper does not describe the same hierarchical intra-cloud consensus mechanism (Liu et al., 5 Mar 2026).

4. Convergence properties of the personalization-oriented scheme

For the 2020 formulation, the composite objective is

$f_i(w_i)=\mathbb{E}_{\xi\sim D_i}[\ell(w_i;\xi)]$9

The analysis assumes that each $i$0 is $i$1-smooth:

$i$2

and that at each round only a random subset $i$3 of clients of average size $i$4 participates, with sampling independent across rounds (Wu et al., 2020).

The main convergence result is a sublinear stationarity guarantee. If the step sizes satisfy $i$5 and $i$6, then the iterates of synchronous FedBCGD satisfy

$i$7

where

$i$8

and $i$9 is the infimum of $D_i$0 (Wu et al., 2020). The paper states that, in particular, driving the expected squared norm of the block gradient below $D_i$1 requires $D_i$2 rounds.

The proof sketch proceeds through descent inequalities for each block update. For the local step,

$D_i$3

A similar descent bound holds for the $D_i$4-update, and summation over rounds yields the stated rate (Wu et al., 2020). For the asynchronous cloud, a consensus-error term decaying as $D_i$5 per local gossip iteration, with $D_i$6, is added and absorbed into the same $D_i$7 rate once a fixed number of inner gossip steps is performed between each client-to-cloud round (Wu et al., 2020).

The theoretical message is therefore specific: the method provides nonconvex stationarity guarantees for personalized federated optimization with partial participation and an asynchronous cloud layer, rather than communication-complexity guarantees of the $D_i$8 type associated with parameter blocking.

5. Accelerated block-communication FedBCGD+: parameter partitioning, control variates, and SVRG

The 2026 FedBCGD+ addresses a different bottleneck: the high overhead expenses in each communication round for large-scale models such as Vision Transformer (Liu et al., 5 Mar 2026). The method splits model parameters into several blocks, including a shared block, and each client uploads a specific parameter block. The paper states that this is the first work on parameter block communication for training large-scale deep models (Liu et al., 5 Mar 2026).

The algorithmic structure is two-tiered. At the server, $D_i$9 clients are sampled and partitioned into $z$0 disjoint blocks $z$1 of size $z$2. The server broadcasts the global model $z$3 and global variate $z$4 to all selected clients (Liu et al., 5 Mar 2026). At the client side, for each block $z$5 and client $z$6, local updates take the form

$z$7

This combines a minibatch stochastic gradient, a client-drift control term, and an SVRG-style correction (Liu et al., 5 Mar 2026).

The client-drift control variate is explicitly connected to SCAFFOLD. Each client $z$8 keeps $z$9, and the global control variate is

$w_i$0

The server updates the blockwise control variates by

$w_i$1

where $w_i$2 and $w_i$3 (Liu et al., 5 Mar 2026). The paper states that this ensures $w_i$4, so adding $w_i$5 corrects local drift.

The variance-reduction component is the zero-mean correction

$w_i$6

motivated by the identity $w_i$7 (Liu et al., 5 Mar 2026). The local descent direction is summarized as

$w_i$8

followed by $w_i$9 (Liu et al., 5 Mar 2026).

At the server, each block is aggregated in parallel:

$i$0

followed by

$i$1

The shared block $i$2 is aggregated similarly (Liu et al., 5 Mar 2026). Unlike the 2020 scheme, the block structure here is over parameter coordinates and explicitly targets reduction of transmitted floating-point values.

6. Rates and communication complexity in the block-communication line

The 2026 paper provides convergence theorems under smoothness and either strong convexity, general convexity, or non-convex assumptions. Let $i$3 denote the number of clients, $i$4 the number of sampled clients, $i$5, $i$6 the number of blocks, and define the effective server step $i$7 (Liu et al., 5 Mar 2026).

In the strongly convex case, if $i$8, then after $i$9 rounds,

$\mu>0$0

and to reach $\mu>0$1 one needs

$\mu>0$2

with communication cost

$\mu>0$3

In the general convex case, under $\mu>0$4,

$\mu>0$5

so $\mu>0$6-error takes

$\mu>0$7

rounds and

$\mu>0$8

In the non-convex case, if $\mu>0$9 and $w_1,\dots,w_N \in \mathbb{R}^d$00, then

$w_1,\dots,w_N \in \mathbb{R}^d$01

so

$w_1,\dots,w_N \in \mathbb{R}^d$02

and

$w_1,\dots,w_N \in \mathbb{R}^d$03

(Liu et al., 5 Mar 2026).

The paper attributes the $w_1,\dots,w_N \in \mathbb{R}^d$04 improvement to the fact that only two blocks—the client's assigned block and a small shared block—of size $w_1,\dots,w_N \in \mathbb{R}^d$05 are sent per client, so the per-round cost is $w_1,\dots,w_N \in \mathbb{R}^d$06 instead of $w_1,\dots,w_N \in \mathbb{R}^d$07 (Liu et al., 5 Mar 2026). In the communication-complexity discussion, FedAvg and SCAFFOLD are described as sending a full $w_1,\dots,w_N \in \mathbb{R}^d$08-vector, namely $w_1,\dots,w_N \in \mathbb{R}^d$09 or $w_1,\dots,w_N \in \mathbb{R}^d$10 floats per client, whereas FedBCGD⁺ sends only two blocks out of $w_1,\dots,w_N \in \mathbb{R}^d$11, for a total of $w_1,\dots,w_N \in \mathbb{R}^d$12 floats per client (Liu et al., 5 Mar 2026).

The following comparison summarizes only quantities stated explicitly in the source material.

Variant	Core objective	Communication emphasis
FedBCGD⁺ in (Wu et al., 2020)	Learn a global model $w_1,\dots,w_N \in \mathbb{R}^d$13 and personalized models $w_1,\dots,w_N \in \mathbb{R}^d$14 via a quadratic penalty	Hierarchical cloud↔edge and intra-cloud protocols, including sync and async settings
FedBCGD+ in (Liu et al., 5 Mar 2026)	Communication-efficient training via parameter blocks, client-drift control, and SVRG-style correction	Per-round upload of $w_1,\dots,w_N \in \mathbb{R}^d$15 floats and communication complexity lower by a factor $w_1,\dots,w_N \in \mathbb{R}^d$16

A plausible implication is that the two papers use the same label to denote different optimizations of the federated-learning stack: one emphasizes personalization and latency under hierarchical coordination, while the other emphasizes parameter-block communication efficiency for large models.

7. Empirical findings and relation to neighboring methods

The 2020 paper reports that FedBCGD⁺ combines momentum on $w_1,\dots,w_N \in \mathbb{R}^d$17 and adaptive penalty $w_1,\dots,w_N \in \mathbb{R}^d$18. The momentum update is

$w_1,\dots,w_N \in \mathbb{R}^d$19

with $w_1,\dots,w_N \in \mathbb{R}^d$20, and the adaptive penalty increases $w_1,\dots,w_N \in \mathbb{R}^d$21 from $w_1,\dots,w_N \in \mathbb{R}^d$22 in early rounds to $w_1,\dots,w_N \in \mathbb{R}^d$23 by a geometric schedule (Wu et al., 2020). The paper states that the momentum term often cuts the required communication rounds by 20–30%. On the CIFAR-10 non-IID partition, FedBCGD⁺ achieves 70% accuracy in just 25 rounds, versus approximately 60 rounds for FedAvg, and attains a final personalization accuracy of 83.2%, versus 80.5% for FedAvg (Wu et al., 2020). On EMNIST, Table 2 is described as confirming that FedBCGD⁺ requires 40% fewer rounds to reach 99% personalized-accuracy than any previous method (Wu et al., 2020).

The 2026 paper reports experiments across CIFAR-10, CIFAR-100, Tiny-ImageNet, and EMNIST, and on architectures from LeNet-5 up to ViT-Base (Liu et al., 5 Mar 2026). It states that FedBCGD⁺ and its simpler FedBCGD variant reduce uplink by a factor of $w_1,\dots,w_N \in \mathbb{R}^d$24, accelerate convergence by approximately $w_1,\dots,w_N \in \mathbb{R}^d$25–$w_1,\dots,w_N \in \mathbb{R}^d$26 versus FedAvg and SCAFFOLD, and match or improve final accuracy under non-IID splits (Liu et al., 5 Mar 2026). In the excerpt from Table 2, for LeNet-5 on CIFAR-100 with 100 clients, $w_1,\dots,w_N \in \mathbb{R}^d$27, and total budget $w_1,\dots,w_N \in \mathbb{R}^d$28, FedAvg reaches 40% test accuracy in $w_1,\dots,w_N \in \mathbb{R}^d$29 of upload, while FedBCGD⁺ reaches the same 40% in only $w_1,\dots,w_N \in \mathbb{R}^d$30 (Liu et al., 5 Mar 2026). In the excerpt from Table 5, for ViT-Base on Tiny ImageNet with $w_1,\dots,w_N \in \mathbb{R}^d$31 and 6 blocks, FedAvg needs $w_1,\dots,w_N \in \mathbb{R}^d$32 to hit 70% accuracy, FedBCGD achieves 83.5% in $w_1,\dots,w_N \in \mathbb{R}^d$33, and FedBCGD⁺ hits 81.3% in $w_1,\dots,w_N \in \mathbb{R}^d$34 (Liu et al., 5 Mar 2026).

The neighboring baselines mentioned explicitly are FedAvg, SCAFFOLD, and, in the 2020 paper, the state of the art for personalized models on edge devices (Wu et al., 2020, Liu et al., 5 Mar 2026). A common misconception would be to treat FedBCGD⁺ as a single method directly comparable across all these reported metrics. The source material indicates instead that the 2020 and 2026 versions solve different problems and report different types of gains. One emphasizes faster convergence for personalized models and latency reduction under asynchronous cloud aggregation; the other emphasizes reduced communication overhead through parameter-block transmission and accelerated convergence in terms of total transmitted floats (Wu et al., 2020, Liu et al., 5 Mar 2026).

Taken together, the two uses of FedBCGD+ define a small but technically heterogeneous family of federated block-coordinate methods. One branch is anchored in the optimization of coupled global and personalized models with a quadratic penalty and hierarchical communication. The other is anchored in blockwise parameter communication, server momentum, client-drift control, and SVRG-style local correction. Their commonality lies in block-coordinate structure; their divergence lies in what constitutes a block, which bottleneck is being optimized, and which convergence and efficiency criteria are foregrounded.