FedBCGD: Federated Block Coordinate Gradient Descent

• FedBCGD is a federated optimization method that partitions model parameters into blocks and transmits only selected updates each round to minimize communication load.
• It employs alternating block updates between client-specific and global variables using gradient descent and, in some cases, exact minimization for personalization.
• Empirical studies demonstrate significant communication savings and faster convergence on large-scale models compared to full-model update techniques.

Searching arXiv for FedBCGD and closely related federated block-coordinate methods. Federated Block Coordinate Gradient Descent (FedBCGD) denotes a class of federated optimization methods that reduce communication by partitioning model parameters or optimization variables into blocks and updating or transmitting only selected blocks in each round, rather than a full model. In the literature, the term covers more than one technical lineage. One line, represented by personalized federated optimization with alternating client and server variables, is a federated block coordinate descent scheme in which local personalized blocks and global cloud blocks are optimized alternately, sometimes by exact minimization on one block and gradient-type updates on another (Wu et al., 2020). A second line uses parameter block communication for large-scale horizontal federated learning: clients train the full model locally but upload only one assigned parameter block and a shared block, with server-side momentum or variance-reduced corrections; this line is explicitly named FedBCD and FedBCGD in recent work (Liu et al., 5 Mar 2026). A third line studies block-coordinate training for vertical or semi-decentralized federated learning, where blocks correspond to feature-partitioned client parameters and communication is organized through stale local block updates or token-based roaming protocols (Liu et al., 2019, Valdeira et al., 2023). Across these variants, the common principle is block-structured federated optimization under communication constraints; the main differences concern whether blocks are model partitions, client-specific personalized variables, or feature-local parameter subsets, and whether the update is exact minimization, gradient descent, or a hybrid of the two.

1. Concept and scope

FedBCGD is most precisely associated with methods that split the model into several parameter blocks and communicate only a subset of those blocks in each round. In the horizontal federated learning setting studied by "FedBCD:Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learning" (Liu et al., 5 Mar 2026), the global parameter vector is partitioned as

$$\boldsymbol{x} = \big[ \boldsymbol{x}_{(1)}^\top,\ldots,\boldsymbol{x}_{(N)}^\top,\boldsymbol{x}_s^\top \big]^\top,$$

where $\boldsymbol{x}_{(j)}$ is the $j$-th parameter block and $\boldsymbol{x}_s$ is a shared block transmitted by all participating clients, typically the final classifier layer. In each communication round, the server samples $S=N\cdot K$ clients, splits them into $N$ groups, and assigns one ordinary block to each group. Every selected client trains the full model locally but uploads only one assigned block and the shared block (Liu et al., 5 Mar 2026). This is not gradient compression in the usual sense; it is a structured parameter-block communication scheme.

The broader literature also uses closely related terminology for methods that alternate over federated blocks even when not every block is updated by a gradient step. In "Federated Block Coordinate Descent Scheme for Learning Global and Personalized Models" (Wu et al., 2020), the variables are partitioned into personalized client models $x_i$ and a global model $z$, and optimization proceeds by alternating local and cloud-side block updates. The paper is not a vanilla block-coordinate gradient method in the narrow sense because the cloud block is typically updated by exact minimization, whereas local blocks may be updated exactly or approximately by gradient-style iterations. This suggests that FedBCGD is often best understood as an umbrella label whose concrete instantiations range from pure block-gradient methods to hybrid block coordinate descent schemes (Wu et al., 2020).

In vertical federated learning, the block interpretation is different. "A Communication Efficient Collaborative Learning Framework for Distributed Features" (Liu et al., 2019) studies distributed-feature learning in which each party owns a block of features and the corresponding local parameter block $\theta_k$. Its FedBCD method performs stochastic partial-gradient updates on one party’s parameter block while using stale cross-party information during multiple local steps between synchronizations. "A Multi-Token Coordinate Descent Method for Semi-Decentralized Vertical Federated Learning" (Valdeira et al., 2023) likewise treats client feature partitions as blocks and updates selected client blocks via token-based coordinate-gradient steps. In this setting, FedBCGD refers less to blockwise model transmission and more to blockwise optimization induced by feature partitioning.

2. Horizontal federated block communication

The most explicit modern formulation of FedBCGD appears in (Liu et al., 5 Mar 2026). The paper studies horizontal federated learning with a central server and $M$ clients, each storing private local data

$\boldsymbol{x}_{(j)}$0

and the standard global objective

$\boldsymbol{x}_{(j)}$1

Its motivation is communication asymmetry: upload bandwidth is emphasized as especially limited, and for large-scale architectures such as Vision Transformers, transmitting the full model each round can dominate training cost (Liu et al., 5 Mar 2026).

The key algorithmic design is to partition the model into $\boldsymbol{x}_{(j)}$2 ordinary parameter blocks plus one shared block. In round $\boldsymbol{x}_{(j)}$3, the server samples $\boldsymbol{x}_{(j)}$4 clients, partitions them into $\boldsymbol{x}_{(j)}$5 groups of size $\boldsymbol{x}_{(j)}$6, sends the current full model $\boldsymbol{x}_{(j)}$7 to all selected clients, and assigns block $\boldsymbol{x}_{(j)}$8 to group $\boldsymbol{x}_{(j)}$9. Each client performs $j$0 local SGD updates on the whole model,

$j$1

but after local training uploads only

$j$2

The server aggregates each ordinary block across the clients assigned to that block,

$j$3

and averages the shared block across all participating clients,

$j$4

It then applies server-side momentum

$j$5

before reconstructing the full model (Liu et al., 5 Mar 2026).

A noteworthy negative result in this paper concerns a freezing variant. The authors considered updating only the assigned block and shared block locally while freezing the rest, but report that it converges poorly because of large drift between updated and frozen blocks. The adopted FedBCGD design therefore trains all parameters locally while communicating only a subset (Liu et al., 5 Mar 2026). This distinction separates parameter-block communication from partial local optimization.

The paper further introduces FedBCGD+, which augments the basic block communication scheme with client drift control and stochastic variance reduction. Its local update is

$j$6

The paper explicitly compares the resulting method to SCAFFOLD-like drift correction and SVRG-style variance reduction, but within a block-communication setting rather than full-model communication (Liu et al., 5 Mar 2026).

3. Personalized and hierarchical federated formulations

A distinct but closely related notion of federated block-coordinate optimization appears in (Wu et al., 2020). Rather than splitting a single model into parameter blocks for communication, the paper formulates personalized federated learning as joint optimization over client-specific models $j$7 and a global model $j$8: $j$9 or, with weights $\boldsymbol{x}_s$0,

$\boldsymbol{x}_s$1

Here $\boldsymbol{x}_s$2 is the personalized client model, $\boldsymbol{x}_s$3 is the global server model, and $\boldsymbol{x}_s$4 controls the degree of personalization: larger $\boldsymbol{x}_s$5 pushes local models toward a shared global model, while smaller $\boldsymbol{x}_s$6 permits stronger local adaptation (Wu et al., 2020).

The block structure is explicit. The algorithm alternates between local blocks $\boldsymbol{x}_s$7 and the global block $\boldsymbol{x}_s$8. Given $\boldsymbol{x}_s$9, each client solves or approximately solves

$S=N\cdot K$0

A basic gradient realization is

$S=N\cdot K$1

possibly repeated multiple times locally. Then the cloud block is updated by exact minimization,

$S=N\cdot K$2

whose solution is the average

$S=N\cdot K$3

or the weighted average

$S=N\cdot K$4

This exact cloud-side block update is a defining structural difference from FedAvg, because averaging is the exact minimizer of the quadratic coupling subproblem rather than a heuristic aggregation rule (Wu et al., 2020).

The same paper extends the setting to a hierarchical cloud-edge architecture. In the asynchronous realization, the single global variable $S=N\cdot K$5 is replaced by cloud-side variables $S=N\cdot K$6, each associated with a cloud node, with edge devices attached through an assignment $S=N\cdot K$7: $S=N\cdot K$8 A consensus mechanism among the $S=N\cdot K$9's is then implemented algorithmically. A generic consensus-plus-innovation form consistent with the paper’s description is

$N$0

where the innovation term is induced by currently available edge-side models. The paper characterizes this as an asynchronous protocol inspired by multi-agent consensus technique and argues that it has the potential for large gains in latency when edge-device updates are intermittent (Wu et al., 2020).

This personalized formulation is not a canonical instance of block-coordinate gradient descent in the narrow all-gradient sense, because the cloud step is typically exact minimization. Nevertheless, it is a federated block coordinate method in which the variables are partitioned across client and server tiers, and it is often treated as conceptually adjacent to FedBCGD (Wu et al., 2020).

4. Vertical and semi-decentralized variants

In vertical federated learning, FedBCGD-like methods arise because data are partitioned by features rather than by samples. "A Communication Efficient Collaborative Learning Framework for Distributed Features" (Liu et al., 2019) considers $N$1 parties that share users but hold disjoint feature subsets $N$2, with one party holding the labels. The objective is

$N$3

For additive models, the loss takes the form

$N$4

with local contributions

$N$5

The stochastic partial gradient is

$N$6

FedBCD then performs blockwise stochastic-gradient updates

$N$7

using exchanged intermediate values $N$8 rather than raw data or model parameters (Liu et al., 2019).

The central communication mechanism is delayed synchronization. In the parallel version FedBCD-p, each party performs $N$9 local updates on its own block using stale cross-party information from the last synchronization. Under suitable choices $x_i$0 and $x_i$1, the paper proves that within $x_i$2 local iterations the method achieves $x_i$3 accuracy measured by the average gradient norm squared while requiring only $x_i$4 communication rounds (Liu et al., 2019). This is a distinct FedBCGD paradigm: communication reduction comes from multiple local block-gradient steps between synchronizations rather than from partial parameter upload.

Semi-decentralized vertical federated learning extends this idea further. In "A Multi-Token Coordinate Descent Method for Semi-Decentralized Vertical Federated Learning" (Valdeira et al., 2023), each client $x_i$5 holds a local model $x_i$6, and a fusion model $x_i$7 acts on the aggregated representation

$x_i$8

The token

$x_i$9

contains enough global state to compute local block gradients. At client $z$0, the coordinate step is

$z$1

with all other blocks fixed. Tokens then roam across a client graph according to a Markov chain, and multiple such tokens can operate in parallel before synchronization through a server (Valdeira et al., 2023). This method is best understood as a parallel Markov chain block coordinate descent algorithm specialized to semi-decentralized vertical federated learning.

5. Convergence results and communication claims

The theoretical guarantees in the FedBCGD literature depend strongly on the variant. The communication-efficient horizontal FedBCGD paper (Liu et al., 5 Mar 2026) assumes strong convexity or convexity, smoothness, unbiased stochastic gradients with bounded variance, bounded heterogeneity, and stochastic gradient smoothness for the variance-reduced analysis. Under these assumptions, the paper states communication complexity results showing that the proposed algorithms reduce the communication term by roughly a factor $z$2, where $z$3 is the number of parameter blocks. It summarizes the transmitted floats per round as $z$4 for FedAvg, $z$5 for FedBCGD, and $z$6 for FedBCGD+, with the factor $z$7 in FedBCGD+ arising from sending both parameters and control-variate increments (Liu et al., 5 Mar 2026).

For the strongly convex case, the same paper gives for FedBCGD+ the communication complexity

$z$8

and when choosing $z$9,

$\theta_k$0

For non-convex objectives it reports

$\theta_k$1

and explicitly states that the factor-$\theta_k$2 reduction is the headline structural claim (Liu et al., 5 Mar 2026).

The personalized hierarchical method (Wu et al., 2020) states that it characterizes the theoretical convergence rate of the algorithm and that the asynchronous protocol can greatly reduce latency when edge updates are intermittent. The exact theorem statements are not reproduced in the provided material, but the paper is described as establishing convergence under standard smoothness, convexity, and asynchronous-consensus assumptions. This suggests a convergence theory grounded in block-coordinate analysis but extended to cloud-edge hierarchy and intermittent updates (Wu et al., 2020).

For vertical federated learning, (Liu et al., 2019) provides an explicit nonconvex stationarity result. Under Lipschitz gradient and sampling assumptions, if the stepsize satisfies a condition depending on $\theta_k$3 and the block Lipschitz constants, then

$\theta_k$4

is bounded by three terms: an optimization term decreasing as $\theta_k$5, a stale-gradient penalty scaling like $\theta_k$6, and a variance term scaling like $\theta_k$7. With $\theta_k$8 and $\theta_k$9, the resulting accuracy is $M$0 with only $M$1 communication rounds (Liu et al., 2019).

The semi-decentralized MTCD paper (Valdeira et al., 2023) gives $M$2 convergence rates in average squared gradient norm. For nonconvex objectives, this is proved in a token-per-cluster setting where tokens roam over disjoint subsets of clients; for convex objectives, the same rate is obtained when tokens roam over possibly overlapping subsets (Valdeira et al., 2023). The step-size bounds depend explicitly on smoothness $M$3, roaming length $M$4, local updates $M$5, and graph coverage constants.

As a centralized theoretical background rather than a federated method, "Iteration Complexity Analysis of Block Coordinate Descent Methods" (Hong et al., 2013) establishes global sublinear $M$6 iteration complexity for a broad BSUM family including BCGD and BCPG, and an accelerated $M$7 rate in a special two-block Gauss–Seidel setting. This does not prove federated convergence directly, but it supplies a reference complexity baseline for block coordinate gradient methods more generally (Hong et al., 2013).

6. Empirical behavior, distinctions, and limitations

The empirical results reported for horizontal FedBCGD emphasize communication efficiency on large models. On CIFAR-100 with LeNet-5, FedBCGD reaches 40% accuracy with $M$8 communication floats versus FedAvg’s $M$9, reported as about $\boldsymbol{x}_{(j)}$00 speedup. On ResNet-18 and CIFAR-100, FedBCGD+ reaches 54% accuracy with $\boldsymbol{x}_{(j)}$01 versus FedBCGD’s $\boldsymbol{x}_{(j)}$02, a reported $\boldsymbol{x}_{(j)}$03 speedup (Liu et al., 5 Mar 2026). For ViT-Base with $\boldsymbol{x}_{(j)}$04 and total communication $\boldsymbol{x}_{(j)}$05, the paper reports on CIFAR-100 that FedBCGD achieves $\boldsymbol{x}_{(j)}$06 with target reached in $\boldsymbol{x}_{(j)}$07, compared with FedAvg’s $\boldsymbol{x}_{(j)}$08 and $\boldsymbol{x}_{(j)}$09; on Tiny ImageNet it reports $\boldsymbol{x}_{(j)}$10 with $\boldsymbol{x}_{(j)}$11, versus FedAvg’s $\boldsymbol{x}_{(j)}$12 and $\boldsymbol{x}_{(j)}$13 (Liu et al., 5 Mar 2026). The paper also states that the freezing variant is harmful, that a shared block improves both speed and final accuracy, and that server momentum significantly accelerates convergence (Liu et al., 5 Mar 2026).

In personalized federated learning, (Wu et al., 2020) reports experiments showing faster convergence for personalized models on edge devices compared to the state of the art, along with latency gains for the asynchronous hierarchical protocol when client participation is intermittent. Because the provided material does not list exact datasets or numerical values, the most defensible empirical summary is that the paper claims faster convergence for personalized models and systems-level latency advantages rather than only iteration-level gains (Wu et al., 2020).

In vertical federated learning, (Liu et al., 2019) reports large reductions in communication rounds. On MIMIC-LR, target AUC 84% is reached in 334 rounds for FedSGD, 71 rounds for FedBCD-p with $\boldsymbol{x}_{(j)}$14, and 52 rounds for FedBCD-p with $\boldsymbol{x}_{(j)}$15. On MNIST-CNN, target AUC 99.7% is reached in 46 rounds for FedSGD, 16 rounds for FedBCD-p with $\boldsymbol{x}_{(j)}$16, and 8 rounds for FedBCD-p with $\boldsymbol{x}_{(j)}$17 (Liu et al., 2019). The same paper reports that too-large $\boldsymbol{x}_{(j)}$18 can degrade convergence, and introduces a proximal modification for stabilization at large $\boldsymbol{x}_{(j)}$19, indicating an explicit communication-versus-drift trade-off (Liu et al., 2019).

The semi-decentralized MTCD work (Valdeira et al., 2023) reports improved communication efficiency over state-of-the-art communication baselines in vertical federated learning and highlights that the method provides a tunable amount of parallel communication. The paper frames its single-token limit as more communication-efficient than decentralized consensus-type baselines but slower per iteration, and presents multi-token operation as a way to mitigate graph-induced cover-time bottlenecks (Valdeira et al., 2023).

Several distinctions recur across the literature. First, FedBCGD should not be conflated with gradient compression or sparsification. The horizontal block-communication method (Liu et al., 5 Mar 2026) explicitly distinguishes itself from Top-$\boldsymbol{x}_{(j)}$20, Rand-$\boldsymbol{x}_{(j)}$21, FedPAQ, and QSGD, arguing that those methods compress already computed full updates, whereas FedBCGD changes what is communicated structurally by assigning semantically defined parameter blocks. Second, FedBCGD differs from methods that locally update only a subset of variables. In (Liu et al., 5 Mar 2026), all variables are trained locally even though only one block is uploaded. Third, some methods described as FedBCD are closer to block coordinate gradient descent than to classical exact coordinate descent. This is true both for the vertical distributed-feature method (Liu et al., 2019) and for token-based semi-decentralized VFL (Valdeira et al., 2023).

A final limitation is terminological. The literature uses FedBCD, FedBCGD, federated block coordinate descent, and related names inconsistently. Some methods update every block by stochastic gradients; some alternate exact minimization on one block with gradient descent on another; some are communication-centric and split model parameters into upload blocks; others are optimization-centric and arise from feature or personalization structure. This suggests that FedBCGD is best treated as a family of federated block-coordinate first-order methods rather than a single canonical algorithm. Under that broader interpretation, the main unifying idea remains stable: exploit block structure to reduce communication, preserve privacy constraints, and align the optimization algorithm with the natural partitioning of variables across clients, servers, or tiers of a federated system (Wu et al., 2020, Liu et al., 5 Mar 2026, Liu et al., 2019, Valdeira et al., 2023).