--- title: 'FedBCGD+: Federated Block Coordinate Descent' url: https://www.emergentmind.com/topics/fedbcgd type: topic --- # FedBCGD+: Federated Block Coordinate Descent Searching arXiv for the cited FedBCGD/FedBCGD+ papers and closely related terminology. arXiv search query: "FedBCGD+ federated block coordinate descent" FedBCGD+ is a name used in arXiv federated-learning literature for two distinct block-coordinate methods. In "Federated Block Coordinate Descent Scheme for Learning Global and Personalized Models" [2012.13900], FedBCGD⁺ denotes a federated block-coordinate-descent algorithm for joint learning of a global model and personalized local models under a quadratic penalty, together with a hierarchical cloud-to-edge and intra-cloud communication structure. In "FedBCD: Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learning" [2603.05116], FedBCGD+ denotes an accelerated communication-efficient method that partitions parameters into blocks, transmits only selected blocks, and augments client updates with client-drift control and stochastic variance reduction. The shared label reflects a common block-coordinate perspective, but the optimization objectives, communication models, and theoretical emphases differ substantially. ## 1. Terminology and problem setting In the 2020 formulation, the central objective is personalization under federated privacy constraints. The method minimizes a coupled objective over a global model $z \in \mathbb{R}^d$ and client-specific models $w_1,\dots,w_N \in \mathbb{R}^d$: $$ \min_{z \in \mathbb{R}^d,\;\{w_i\}\subset \mathbb{R}^d} F(z,\{w_i\}) \;\triangleq\; \sum_{i=1}^N f_i(w_i) + \frac{\mu}{2}\sum_{i=1}^N \|w_i-z\|^2. $$ Here $f_i(w_i)=\mathbb{E}_{\xi\sim D_i}[\ell(w_i;\xi)]$ is the expected loss on device $i$'s local distribution $D_i$, $z$ is the global consensus model, $w_i$ is device $i$'s personalized model, and $\mu>0$ is a regularization hyperparameter often called the personalization weight [2012.13900]. In the 2026 formulation, the central objective is communication efficiency for large-scale models such as Vision Transformer. The method splits model parameters into several blocks, including a shared block, and enables uploading a specific parameter block by each client. FedBCGD+ extends the basic FedBCGD framework with a client-drift control variate $c_i$ and an SVRG-style variance-reduction correction [2603.05116]. A plausible implication is that the term "FedBCGD+" should not be treated as a single canonical algorithm without reference to its source paper. In the 2020 line, "plus" refers to momentum on the global model and an adaptive penalty schedule; in the 2026 line, "plus" refers to an accelerated block-coordinate scheme with client drift control and stochastic variance reduction [2012.13900; 2603.05116]. ## 2. Personalized FedBCGD⁺: joint global and local models The 2020 method begins from a quadratic-penalty formulation that couples a global template to personalized client models. The penalty term $\frac{\mu}{2}\sum_i \|w_i-z\|^2$ tethers each $w_i$ to $z$, so that learning is neither purely global nor fully decoupled [2012.13900]. The paper states that by driving $\mu \to \infty$ one recovers pure personalization $(w_i=z)$, whereas $\mu \to 0$ decouples the clients and each solves its own problem. In practice, $\mu$ is chosen to trade off global generalization against local tailoring [2012.13900]. The synchronous algorithm is organized as a two-step block-coordinate procedure. At communication round $k$, the cloud broadcasts the current global model $z^k$ to a subset $S^k \subseteq \{1,\dots,N\}$ of participating clients. Each client $i \in S^k$ updates its local block while holding $z^k$ fixed: $$ w_i^{k+1}=\arg\min_w f_i(w)+\frac{\mu}{2}\|w-z^k\|^2+\frac{1}{2\eta_w}\|w-w_i^k\|^2. $$ The implementation uses a single gradient step, $$ w_i^{k+1}=w_i^k-\eta_w\big[\nabla f_i(w_i^k)+\mu(w_i^k-z^k)\big], $$ with step size $\eta_w \le 1/(L_i+\mu)$ [2012.13900]. The cloud then updates the global block by solving $$ z^{k+1}=\arg\min_z \frac{\mu}{2}\sum_{i\in S^k}\|w_i^{k+1}-z\|^2+\frac{1}{2\eta_z}\|z-z^k\|^2, $$ which yields the gradient-style update $$ z^{k+1}=z^k-\eta_z\mu\sum_{i\in S^k}(z^k-w_i^{k+1}) =(1-\gamma)z^k+\frac{\gamma}{|S^k|}\sum_{i\in S^k} w_i^{k+1}, $$ where $\gamma=\eta_z \mu |S^k|$ [2012.13900]. For the special choice $\eta_z=1/(\mu |S^k|)$, the update reduces to the model-averaging rule $$ z^{k+1}=\frac{1}{|S^k|}\sum_{i\in S^k} w_i^{k+1}. $$ This formulation makes block structure explicit at the level of optimization variables rather than parameter partitions. One block is the shared global template $z$, and the other blocks are the personalized client models $w_i$. This suggests that the method is aimed at statistically heterogeneous data settings in which a single fully shared model is not adequate. ## 3. Hierarchical communication and asynchronous cloud operation A defining feature of the 2020 FedBCGD⁺ line is its two-layer communication structure: cloud-to-edge and intra-cloud among sub-cloud servers [2012.13900]. In large-scale deployment, the cloud may itself be geographically distributed. The paper arranges cloud servers in a fast connectivity graph $G$ and runs a lightweight consensus (gossip) or push-sum protocol. The asynchronous procedure is described concretely. Whenever one server receives an updated $w_i$ from edge $i$, it incorporates it into its local sum, asynchronously exchanges partial sums with its neighbors in $G$, and applies a local $z$-update step of the same form as the synchronous global update but with the partially aggregated average [2012.13900]. Under mild conditions—connected $G$ and proper consensus weights—the stale aggregates converge to the true average exponentially fast in the intra-cloud iteration count. The paper explicitly frames this asynchronous protocol as being inspired by multi-agent consensus technique and states that it has the potential for large gains in latency compared to a synchronous setting when the edge-device updates are intermittent [2012.13900]. The significance of this design is not merely reduced idle time at slow clients. It also moves part of the aggregation burden into a distributed cloud layer, which is distinct from standard single-server federated averaging. This architecture should not be conflated with the 2026 block-communication method. In the latter, block structure refers to partitioned parameter coordinates, and the server aggregates per-parameter-block updates in parallel. The 2026 paper does not describe the same hierarchical intra-cloud consensus mechanism [2603.05116]. ## 4. Convergence properties of the personalization-oriented scheme For the 2020 formulation, the composite objective is $$ L(z,\{w_i\}) := \sum_{i=1}^N f_i(w_i) + \frac{\mu}{2}\sum_{i=1}^N \|w_i-z\|^2. $$ The analysis assumes that each $f_i$ is $L_i$-smooth: $$ \|\nabla f_i(w)-\nabla f_i(w')\| \le L_i \|w-w'\|,\quad \forall w,w', $$ and that at each round only a random subset $S^k$ of clients of average size $m$ participates, with sampling independent across rounds [2012.13900]. The main convergence result is a sublinear stationarity guarantee. If the step sizes satisfy $\eta_w \le 1/(\max_i(L_i+\mu))$ and $\eta_z \le 1/(\mu N)$, then the iterates of synchronous FedBCGD satisfy $$ \frac{1}{T}\sum_{k=0}^{T-1}\mathbb{E}\Big[\|\nabla_z L(z^k,\{w_i^k\})\|^2 +\sum_{i=1}^N \|\nabla_{w_i}L(z^k,\{w_i^k\})\|^2\Big] \le \frac{L(z^0,\{w_i^0\})-L^*}{cT}, $$ where $$ c=\min\{\mu-\eta_w(L_i+\mu)^2,\; \mu N-\eta_z \mu^2 N^2\}>0, $$ and $L^*$ is the infimum of $L$ [2012.13900]. The paper states that, in particular, driving the expected squared norm of the block gradient below $\epsilon$ requires $O(1/\epsilon)$ rounds. The proof sketch proceeds through descent inequalities for each block update. For the local step, $$ L(z^k,\{w_i^k\})-L(z^k,\{w_i^{k+1}\}) \ge \Big(\frac{1}{2\eta_w}-\frac{L_i+\mu}{2}\Big)\|w_i^{k+1}-w_i^k\|^2. $$ A similar descent bound holds for the $z$-update, and summation over rounds yields the stated rate [2012.13900]. For the asynchronous cloud, a consensus-error term decaying as $O(\rho^t)$ per local gossip iteration, with $\rho<1$, is added and absorbed into the same $O(1/T)$ rate once a fixed number of inner gossip steps is performed between each client-to-cloud round [2012.13900]. The theoretical message is therefore specific: the method provides nonconvex stationarity guarantees for personalized federated optimization with partial participation and an asynchronous cloud layer, rather than communication-complexity guarantees of the $1/N$ type associated with parameter blocking. ## 5. Accelerated block-communication FedBCGD+: parameter partitioning, control variates, and SVRG The 2026 FedBCGD+ addresses a different bottleneck: the high overhead expenses in each communication round for large-scale models such as Vision Transformer [2603.05116]. The method splits model parameters into several blocks, including a shared block, and each client uploads a specific parameter block. The paper states that this is the first work on parameter block communication for training large-scale deep models [2603.05116]. The algorithmic structure is two-tiered. At the server, $S=N\cdot K$ clients are sampled and partitioned into $N$ disjoint blocks $\mathcal K_1,\dots,\mathcal K_N$ of size $K$. The server broadcasts the global model $x^r$ and global variate $c^r$ to all selected clients [2603.05116]. At the client side, for each block $j$ and client $k \in \mathcal K_j$, local updates take the form $$ x_{k,j}^{r,t+1}=x_{k,j}^{r,t} -\eta\,\nabla f_{k,j}(x_{k,j}^{r,t};\zeta) +\eta(c^r-c_{k,j}^r) +\eta\bigl(\nabla f_{k,j}(x^r)-\nabla f_{k,j}(x^r;\zeta)\bigr). $$ This combines a minibatch stochastic gradient, a client-drift control term, and an SVRG-style correction [2603.05116]. The client-drift control variate is explicitly connected to SCAFFOLD. Each client $i$ keeps $c_i^r$, and the global control variate is $$ c^r=\frac{1}{M}\sum_{i=1}^M c_i^r. $$ The server updates the blockwise control variates by $$ c_{(j)}^{r+1}=c_{(j)}^r+\frac{1}{M}\sum_{k\in\mathcal K_j}\Delta c_{k,j,(j)}, $$ where $\Delta c_{k,j,(j)}=c_{k,j,(j)}^+-c_{k,j,(j)}^r$ and $c_{k,j}^+=\nabla_{(j)}f_{k,j}(x^r)$ [2603.05116]. The paper states that this ensures $\mathbb{E}[c_i^r]\approx \nabla f_i(x^r)$, so adding $c^r-c_i^r$ corrects local drift. The variance-reduction component is the zero-mean correction $$ +\eta\bigl(\nabla f_{k,j}(x^r)-\nabla f_{k,j}(x^r;\zeta)\bigr), $$ motivated by the identity $\mathbb{E}_\zeta[\nabla f(x^r;\zeta)] = \nabla f(x^r)$ [2603.05116]. The local descent direction is summarized as $$ g_{k,j}=\nabla f_{k,j}(x_{k,j};\zeta) -(c^r-c_{k,j}^r) +\bigl[\nabla f_{k,j}(x^r)-\nabla f_{k,j}(x^r;\zeta)\bigr], $$ followed by $x_{k,j}\leftarrow x_{k,j}-\eta g_{k,j}$ [2603.05116]. At the server, each block is aggregated in parallel: $$ x_{(j)}^{r+1}=\frac{1}{K}\sum_{k\in\mathcal K_j} x_{k,j,(j)}^{r,T},\qquad v_{(j)}^{r+1}=\lambda v_{(j)}^r+\bigl(x_{(j)}^{r+1}-x_{(j)}^r\bigr), $$ followed by $$ x_{(j)}^{r+1}\gets x_{(j)}^{r+1}+v_{(j)}^{r+1}. $$ The shared block $x_s$ is aggregated similarly [2603.05116]. Unlike the 2020 scheme, the block structure here is over parameter coordinates and explicitly targets reduction of transmitted floating-point values. ## 6. Rates and communication complexity in the block-communication line The 2026 paper provides convergence theorems under smoothness and either strong convexity, general convexity, or non-convex assumptions. Let $M$ denote the number of clients, $S$ the number of sampled clients, $K=S/N$, $N$ the number of blocks, and define the effective server step $\tilde\eta=(1-\lambda)^{-1}\eta T$ [2603.05116]. In the strongly convex case, if $\tilde\eta \le \min\{\frac{1}{81\beta}, \frac{K}{15\mu M}\}$, then after $R$ rounds, $$ \mathbb{E}[f(x^R)]-f^* = O\!\Bigl(\Bigl(\frac{M}{K}+\frac{\beta}{\mu}\Bigr) \exp\!\{-\min(\tfrac{K}{30M},\tfrac{\mu}{162\beta})R\}\Bigr), $$ and to reach $\mathbb{E}[f(x^R)]-f^*\le \epsilon$ one needs $$ R=O\!\Bigl(\Bigl(\frac{M}{K}+\frac{\beta}{\mu}\Bigr)\ln\frac{1}{\epsilon}\Bigr), $$ with communication cost $$ R\times \frac{2d}{N} = O\!\Bigl(\Bigl(\frac{M}{S}+\frac{\beta}{\mu N}\Bigr)d\ln\frac{1}{\epsilon}\Bigr). $$ In the general convex case, under $\tilde\eta \le 1/\beta$, $$ \mathbb{E}[f(x^R)]-f^*= O\Bigl(\sqrt{\frac{M}{K}\,\frac{\beta D^2}{R}}\Bigr), $$ so $\epsilon$-error takes $$ R=O\Bigl(\frac{M\beta D^2}{K\epsilon^2}\Bigr) $$ rounds and $$ \mathrm{Comm.}=O\Bigl(\frac{M}{S}d\frac{\beta D^2}{\epsilon^2}\Bigr)\times \frac{1}{N}. $$ In the non-convex case, if $\tilde\eta \le \frac{1}{24\beta}(K/M)^{2/3}$ and $F=f(x^0)-f^*$, then $$ \min_{r