---
title: Randomised Block Coordinate DR Splitting
url: https://www.emergentmind.com/topics/randomised-block-coordinate-douglas-rachford-splitting
type: topic
---

# Randomised Block Coordinate DR Splitting

Searching arXiv for the cited papers and closely related randomized/block-coordinate Douglas–Rachford work.
Randomised block coordinate Douglas-Rachford splitting denotes a family of Douglas-Rachford (DR) schemes in which only a random subset of blocks is activated at each iteration, typically in a product Hilbert space and often under stochastic perturbations, inexact proximal evaluations, or asynchronous execution. In the operator-theoretic formulation, the method addresses monotone inclusions of the form $0\in A x + B x$ with block structure and i.i.d. random sweeping rules, yielding almost-sure convergence results in Hilbert spaces [1404.7536]. Subsequent developments specialized the same randomized partial-update principle to sparse binary logistic regression through random block-coordinate DR splitting with blockwise preconditioning and a closed-form proximity operator for the logistic loss [1712.09131], to nonconvex federated composite optimization through FedDR and asyncFedDR [2103.03452], and to linear feasibility problems through a randomized $r$-sets DR variant with linear convergence in expectation and heavy-ball acceleration [2207.04291].

## 1. Foundational formulation

The foundational setting is the product Hilbert space
\[
\mathcal H=\mathcal H_1\oplus\mathcal H_2\oplus\cdots\oplus\mathcal H_m,
\]
equipped with two maximally monotone operators
\[
A\colon \mathcal H\to 2^{\mathcal H}, \qquad B\colon \mathcal H\to 2^{\mathcal H},
\]
with block structure
\[
A(x_1,\dots,x_m)= (A_1 x_1,\dots,A_m x_m),\qquad
B(x_1,\dots,x_m)= (B_1(x_1,\dots,x_m),\dots,B_m(x_1,\dots,x_m)).
\]
The associated primal inclusion is
\[
\text{find }x=(x_1,\dots,x_m)\in\mathcal H\quad\text{such that}\quad 0\in A x + B x,
\]
with solution set $\mathsf F=\mathrm{zer}(A+B)\neq\varnothing$ [1404.7536].

The block-coordinate DR viewpoint relies on the resolvents
\[
J_{\gamma A}=(\mathrm{Id}+\gamma A)^{-1}, \qquad J_{\gamma B}=(\mathrm{Id}+\gamma B)^{-1},
\]
where $J_{\gamma A}$ splits blockwise,
\[
J_{\gamma A}(x_1,\dots,x_m)=\bigl(J_{\gamma A_1}x_1,\dots,J_{\gamma A_m}x_m\bigr),
\]
and $J_{\gamma B}(x)=\bigl(Q_1(x),\dots,Q_m(x)\bigr)$ defines the component maps $Q_i\colon\mathcal H\to\mathcal H_i$. This formulation places randomized block activation inside the classical DR resolvent-reflection architecture rather than treating it as an external stochastic wrapper.

| Paper | Problem class | Distinctive randomized feature |
|---|---|---|
| [1404.7536] | $0\in A x+B x$ in product Hilbert spaces | random sweeping rules to select arbitrarily the blocks |
| [1712.09131] | sparse binary logistic regression | randomly selecting a mini-batch of data and update the variables in a block coordinate manner |
| [2103.03452] | nonconvex federated composite optimization | update only a subset of users at each communication round |
| [2207.04291] | feasibility problem derived from linear systems | randomized $r$-sets-DR and heavy ball momentum |

This family should therefore be understood as an operator-splitting framework whose randomized structure is expressed at the level of activated coordinates, samples, users, or reflected sets, depending on the application.

## 2. Random sweeping, block updates, and asymptotic convergence

In the general block-coordinate construction, iteration $n$ activates a random subset of the $m$ blocks through an i.i.d. binary vector
\[
E_n=(E_{1,n},E_{2,n},\dots,E_{m,n})\in\{0,1\}^m\setminus\{0\},
\]
with
\[
p_i=P[E_{i,n}=1]>0,\qquad i=1,\dots,m,
\]
and $E_n$ independent of the past. Additive stochastic errors $a_n,b_n\in\mathcal H$ are permitted under square-integrability conditions. Given $\gamma>0$ and relaxation parameters $\mu_n\in(0,2)$ with $\inf_n\mu_n>0$ and $\sup_n\mu_n<2$, the block-coordinate DR iteration is [1404.7536]
\[
y_n=J_{\gamma B}x_n+b_n,
\]
\[
z_{i,n+1}=z_{i,n}+E_{i,n}\bigl(y_{i,n}-z_{i,n}\bigr),
\]
\[
u_n=J_{\gamma A}(2z_{n+1}-x_n)+a_n,
\]
\[
x_{i,n+1}=x_{i,n}+E_{i,n}\mu_n\bigl(u_{i,n}-z_{i,n+1}\bigr).
\]

The corresponding convergence theorem establishes that, if $\mathsf F=\mathrm{zer}(A+B)\neq\varnothing$, if the sweeping vectors are i.i.d. with positive activation probabilities and independent of the past, if the error moments are finite, and if the relaxation parameters remain in a compact subset of $(0,2)$, then $(x_n)$ converges weakly almost surely to an $\mathsf F$-valued random variable. Under the additional assumptions that $J_{\gamma B}$ is weakly sequentially continuous and $b_n\to0$ almost surely, the auxiliary sequence $(z_n)$ converges strongly almost surely to $z=J_{\gamma B}x\in\mathsf F$, and
\[
\frac{x_n-z_n}{\gamma}\to u
\]
strongly almost surely, where $u$ lies in the set of dual solutions [1404.7536].

No explicit rate is given in that theorem. The significance of the result lies in the stochastic quasi-Fejér framework: random block activation and stochastic perturbations are incorporated directly into a Krasnoselʹskiĭ-Mann analysis of the DR operator composition. When $m=1$, the scheme reduces exactly to the standard relaxed DR iteration with stochastic errors,
\[
z_{n+1}=J_{\gamma B}(x_n)+b_n,\qquad
x_{n+1}=x_n+\mu_n\bigl(J_{\gamma A}(2z_{n+1}-x_n)+a_n-z_{n+1}\bigr),
\]
so the block-coordinate theory contains the classical single-block case as a special instance.

## 3. Sparse binary logistic regression and low-complexity block preconditioning

A concrete specialization appears in sparse logistic regression, where a stochastic version of DR is used to sweep the training set by randomly selecting a mini-batch of data at each iteration and to update variables in a block coordinate manner [1712.09131]. The underlying composite convex problem is posed in product spaces:
\[
\min_{w=(w_1,\dots,w_B)\in H_1\times\cdots\times H_B}
\sum_{b=1}^B f_b(w_b)+\sum_{\ell=1}^L h_\ell\!\left(\sum_{b=1}^B A_{\ell,b}w_b\right),
\]
where $f_b\in\Gamma_0(H_b)$, each $h_\ell$ is convex with $\beta_\ell$-Lipschitz gradient, and $A_{\ell,b}$ are bounded linear operators. Equivalently, the problem can be written as the primal inclusion
\[
0\in \partial f(w)+A^*\nabla h(Aw),
\]
and standard DR splitting is then applied to the maximal-monotone operators $M=\partial f$ and $S=A^*\nabla h A$.

The random block-coordinate stochastic adaptation introduces a mask
\[
\varepsilon^{[i]}=(\varepsilon_1^{[i]},\dots,\varepsilon_B^{[i]},\varepsilon_{B+1}^{[i]},\dots,\varepsilon_{B+L}^{[i]})\in\{0,1\}^{B+L}\setminus\{0\},
\]
with i.i.d. components independent of the past and positive activation probability for every primal and dual block. Possible numerical errors in proximal computations are represented by additive terms $a_b^{[i]}$ and $d_\ell^{[i]}$ satisfying summable-error conditions. The resulting RBC-DRS scheme uses blockwise preconditioners
\[
C_b=\Bigl(\Id+\tau_b\sum_{\ell=1}^L \frac{\gamma_\ell}{1+\gamma_\ell\rho_\ell}A_{\ell,b}^*A_{\ell,b}\Bigr)^{-1},
\]
together with auxiliary quantities
\[
u_b^{[i]}=\sum_{\ell=1}^L \frac{1}{1+\gamma_\ell\rho_\ell}A_{\ell,b}^*s_{\ell,b}^{[i]},
\]
and relaxation $\mu^{[i]}\in(\eta,2-\eta)$.

For binary logistic regression, the sample loss is
\[
h(v)=\log(1+\exp(-v)),\qquad v\in\mathbb R,
\]
with derivative
\[
h'(v)=-\frac{\exp(-v)}{1+\exp(-v)}
\]
and Lipschitz constant $\beta=1/4$. Its proximity operator admits the closed form
\[
\forall v\in\mathbb R,\qquad \prox_{\gamma h}(v)=v+W_{\exp(-v)}\!\bigl(\gamma\exp(-v)\bigr),
\]
where $W_r(\cdot)$ is the generalized Lambert $W$ function solving
\[
w(\exp w+r)=\cdot,
\]
and the branch chosen is the unique nonnegative, increasing one [1712.09131]. As $v\to-\infty$, the expansion
\[
\prox_{\gamma h}(v)=v+\gamma\bigl[1-\exp(v+\gamma)+(1+\gamma)\exp(2(v+\gamma))\bigr]+O(\exp(2v))
\]
is used to avoid overflow in practice.

The convergence statement for RBC-DRS is almost-sure convergence: if the masks are i.i.d., independent of the states, each block has positive activation probability, the deterministic errors satisfy the stated summability conditions, and the step parameters and relaxation are chosen as required, then the sequence $w^{[i]}$ converges almost surely, weakly in each $H_b$, to a solution of the original problem. The low-complexity emphasis is explicit in the per-iteration accounting: the inverses $C_b$ are precomputed once with cost $O(\sum_b N_b^3)$ if $H_b\simeq\mathbb R^{N_b}$; each activated primal block requires one multiplication $C_b\cdot x$ with cost $O(N_b^2)$ and one prox of $f_b$; each activated dual block uses a scalar prox of $h_\ell$; and if on average a fraction $\alpha_{\text{primal}}$ of primal blocks and $\alpha_{\text{dual}}$ of dual blocks is activated, the expected per-iteration cost is
\[
O\!\Bigl(\alpha_{\text{primal}}\sum_b N_b^2+\alpha_{\text{dual}}LB+\alpha_{\text{dual}}\sum_b N_b\Bigr).
\]
The paper’s experiments on standard datasets report efficiency with respect to stochastic gradient-like methods.

## 4. Federated composite optimization: FedDR and asyncFedDR

In federated learning, the same design principle is realized as randomized user activation in a nonconvex composite problem,
\[
\min_{x\in\mathbb R^p} F(x):=\frac1n\sum_{i=1}^n f_i(x)+g(x),
\]
where each $f_i$ is $C^1$ and $L$-smooth, but not necessarily convex, and $g$ is proper, closed, convex and proximable. The target is an $\varepsilon$-stationary point satisfying
\[
\mathbb E\|G_\eta(x)\|^2\le\varepsilon^2,
\qquad
G_\eta(x)=\frac{x-\prox_{\eta g}(x-\eta\nabla f(x))}{\eta}.
\]
At each round $k$, only a random subset $S_k\subseteq\{1,\dots,n\}$ of active clients is chosen, with
\[
P(i\in S_k)=p_i>0
\]
independently of past updates; in practice $S_k$ can be uniform sampling of $B$ clients out of $n$, non-uniform importance sampling, or full participation [2103.03452].

The synchronous FedDR algorithm maintains per-client variables $y_i^k,x_i^k,\hat x_i^k$ and server variables $\tilde x^k,\bar x^k$. For active clients,
\[
y_i^{k+1}=y_i^k+\alpha(\bar x^k-x_i^k),
\]
\[
x_i^{k+1}\approx \prox_{\eta f_i}(y_i^{k+1})\quad\text{within }\|\cdot\|\le \varepsilon_{i,k+1},
\]
\[
\hat x_i^{k+1}=2x_i^{k+1}-y_i^{k+1},
\]
while the server performs
\[
\tilde x^{k+1}=\tilde x^k+\frac1n\sum_{i\in S_k}\Delta \hat x_i^k,
\qquad
\bar x^{k+1}=\prox_{\eta g}(\tilde x^{k+1}).
\]
In compact form, the algorithm is a block-coordinate variant of DR splitting applied to the constrained reformulation $x_1=\cdots=x_n$ with regularizer $g$ on $x_1$. The asynchronous variant, asyncFedDR, lets exactly one client wake up at iteration $k$, read a possibly stale model $\bar x^{k-d^k_{i_k}}$ with delay bounded by $\tau$, perform the same three local steps, and send $\Delta\hat x_{i_k}^k$ to the server, which immediately updates $\tilde x^{k+1}$ and $\bar x^{k+1}$.

The main complexity statements are explicit. Under Assumptions A1–A3 and sufficiently small $\alpha,\eta$, Theorem 3.1 gives
\[
\frac1{K+1}\sum_{k=0}^K\mathbb E\|G_\eta(\bar x^k)\|^2
\le
\frac{C_1[F(x^0)-F^*]}{K+1}
+\frac1{n(K+1)}\sum_{k=0}^K\sum_{i=1}^n\bigl(C_2\varepsilon_{i,k}^2+C_3\varepsilon_{i,k+1}^2\bigr).
\]
With exact prox, $\varepsilon_{i,k}=0$, and $p_i=1/n$, one obtains
\[
\frac1{K+1}\sum_{k=0}^K\mathbb E\|G_{1/(3L)}(\bar x^k)\|^2
\le
\frac{160L\,n\,(F(x^0)-F^*)}{3(K+1)},
\]
hence $O(\varepsilon^{-2})$ rounds to reach $\varepsilon^2$. Under A1, A2, A4, and suitable $\alpha,\eta$, Theorem 3.2 gives for asyncFedDR
\[
\frac1{K+1}\sum_{k=0}^K\mathbb E\|G_\eta(\bar x^k)\|^2
\le
\frac{\hat C[F(x^0)-F^*]}{K+1},
\]
hence also $O(\varepsilon^{-2})$ communication complexity. Both theorems are stated to match the known lower bound up to constants even in the presence of inexact prox’s and asynchrony.

The empirical evidence covers synthetic quadratic-plus-$\ell_2$ problems, FEMNIST, composite $\ell_1$ regularization, and asynchronous MNIST experiments with simulated client delays. The reported findings are that FedDR and FedPD outperform FedProx and FedAvg on synthetic problems, especially in highly non-iid cases; on FEMNIST, FedDR reaches lower loss and higher test-accuracy per byte of communication than FedPD/FedProx/FedAvg; composite $\ell_1$ experiments confirm robustness under inexact prox implemented by local SGD; and asyncFedDR runs up to $35\%$ faster in wall-clock time than synchronous FedDR while preserving convergence speed in rounds.

## 5. Linear feasibility, randomized $r$-sets DR, and heavy-ball acceleration

A related randomized DR construction for linear systems begins from the consistent problem
\[
A x=b,\qquad A\in\mathbb R^{m\times n},\; b\in\mathbb R^m,
\]
which is rewritten as the convex feasibility problem
\[
\text{find }x\in\bigcap_{i=1}^m H_i,\qquad
H_i=\{x\in\mathbb R^n:\langle a_i,x\rangle=b_i\},
\]
where $a_i^\top$ is the $i$th row of $A$ and $b_i$ the $i$th entry of $b$. The orthogonal projector and reflection are
\[
P_{H_i}(x)=x-\frac{\langle a_i,x\rangle-b_i}{\|a_i\|_2^2}\,a_i,
\qquad
R_{H_i}(x)=2P_{H_i}(x)-x
\]
[2207.04291].

The randomized $r$-sets Douglas-Rachford method (RrDR), with relaxation parameter $\alpha\in(0,1)$ and block-size $r\in\mathbb Z_+$, initializes $z_0^k\gets x^k$ and for $\ell=1,\dots,r$ samples
\[
j_{k,\ell}\in\{1,\dots,m\}\quad\text{i.i.d. with}\quad
\Pr\{j_{k,\ell}=i\}=\frac{\|a_i\|_2^2}{\|A\|_F^2},
\]
then applies the reflections
\[
z_\ell^k
=
z_{\ell-1}^k
-2\,
\frac{\langle a_{j_{k,\ell}},z_{\ell-1}^k\rangle-b_{j_{k,\ell}}}
{\|a_{j_{k,\ell}}\|_2^2}
\,a_{j_{k,\ell}},
\]
followed by the relaxed update
\[
x^{k+1}=(1-\alpha)x^k+\alpha z_r^k.
\]

For
\[
x_0^*:=A^\dagger b+(I-A^\dagger A)x^0,
\]
the projection of $x^0$ onto the solution set of $A x=b$, and for $\sigma_{\min}(A)>0$ the smallest nonzero singular value of $A$, the main theorem states
\[
\mathbb E\|x^k-x_0^*\|_2^2
\le
\Bigl[
\alpha^2+(1-\alpha)^2
+2\alpha(1-\alpha)\Bigl(1-2\frac{\sigma_{\min}^2(A)}{\|A\|_F^2}\Bigr)^r
\Bigr]^k
\|x^0-x_0^*\|_2^2.
\]
The contraction factor depends only on the singular-value ratio $\sigma_{\min}^2(A)/\|A\|_F^2$ and the block-size $r$, but not on the ambient dimension $n$. This is significant because the direct extension of DR to more-than-two-sets feasibility, called the $r$-sets-DR method, is stated not to be necessarily convergent, whereas the randomized version brings linear convergence in expectation.

The momentum extension mRrDR adds Polyak heavy-ball momentum,
\[
x^{k+1}=(1-\alpha)x^k+\alpha z_r^k+\beta(x^k-x^{k-1}),\qquad \beta\ge0.
\]
If $\alpha\in(0,1)$ and $\beta\ge0$ satisfy $\gamma_1+\gamma_2<1$, then for $F^k:=\mathbb E\|x^k-x_0^*\|^2$,
\[
F^{k+1}\le \gamma_1 F^k+\gamma_2 F^{k-1}
\quad\Longrightarrow\quad
F^k\le q^k(1+\tau)F^0,
\]
where
\[
q=\frac{\gamma_1+\sqrt{\gamma_1^2+4\gamma_2}}{2}<1,
\qquad
\tau=q-\gamma_1\ge0.
\]
For the norm of the expected iterate, under mild conditions,
\[
\|\bar x^k-x_0^*\|_2^2\le \beta^k C,
\]
which is described as linear decay at rate $\beta<1$, achieving a square-root speedup over the relaxation-only bound. The numerical experiments include synthetic Gaussian systems, SuiteSparse and LIBSVM datasets such as `Franz1` and `ijcnn1`, and randomized gossip or average-consensus systems on cycle, path, and random geometric graphs. The reported empirical findings include faster convergence than randomized Kaczmarz when $r>1$, significant acceleration for mRrDR with $(\alpha,\beta)=(0.5,0.4)$, often $2$–$5\times$ higher efficiency than RK, RGS, and RP-ADMM in row-action count and CPU time, and superiority over MATLAB’s dense-solver routines for large overdetermined consistent systems once $m\gg n$.

## 6. Scope, recurring themes, and common misconceptions

Several recurring themes cut across these works. First, randomised block coordinate DR is not restricted to exact deterministic proximal computation. The Hilbert-space framework explicitly permits additive stochastic errors in the evaluation of the resolvents [1404.7536]. The logistic-regression variant models additive errors $a_b^{[i]}$ and $d_\ell^{[i]}$ under summable-error conditions [1712.09131]. FedDR allows inexact local solves through approximate evaluations of $\prox_{\eta f_i}$, and asyncFedDR additionally tolerates bounded staleness without locking [2103.03452]. A common misconception is therefore that DR-based splitting requires exact full-batch proximal subproblems at every step; the randomized block-coordinate literature shows otherwise.

Second, the framework is not limited to convex finite-dimensional regression. The 2014 block-coordinate theory is formulated for structured monotone inclusion and convex minimization problems in Hilbert spaces [1404.7536]. The 2017 RBC-DRS instantiation targets sparse logistic regression [1712.09131]. FedDR and asyncFedDR treat nonconvex federated composite optimization with convex regularizers [2103.03452]. The randomized $r$-sets DR method addresses linear feasibility induced by linear systems and augments it with heavy-ball momentum [2207.04291]. This suggests that randomized partial activation is best viewed as a structural modification of the DR iteration rather than as an application-specific heuristic.

Third, convergence guarantees depend strongly on the problem class. In the general block-coordinate monotone setting, the available guarantees are weak almost-sure convergence of $(x_n)$ and, under additional continuity and vanishing-error assumptions, strong almost-sure convergence of $(z_n)$, with no explicit rate [1404.7536]. In federated nonconvex optimization, the relevant guarantee is an $O(1/K)$ bound on the average squared gradient mapping norm, hence $O(\varepsilon^{-2})$ communication complexity [2103.03452]. In linear systems, randomization yields linear convergence in expectation, and heavy-ball momentum yields an accelerated rate for the norm of the expected error [2207.04291]. A second common misconception is therefore that all randomized DR variants admit a single universal rate statement; the theory is in fact highly regime-dependent.

Finally, relation to the classical DR method remains direct rather than merely analogical. When $m=1$, the block-coordinate Hilbert-space scheme reduces exactly to the standard relaxed DR iteration with stochastic errors [1404.7536]. The logistic-regression method remains a DR splitting on maximal-monotone operators with random block and mini-batch activation [1712.09131]. FedDR is stated compactly as a block-coordinate variant of DR splitting for the constrained reformulation $x_1=\cdots=x_n$ [2103.03452]. The linear-system RrDR preserves the projector-reflector core of DR while randomizing reflected hyperplanes over $r$ successive draws [2207.04291]. Taken together, these results indicate that randomised block coordinate Douglas-Rachford splitting is a coherent extension of DR methodology in which partial activation, stochastic approximation, and structural decomposition are combined without abandoning the underlying resolvent-reflection geometry.

Source: https://www.emergentmind.com/topics/randomised-block-coordinate-douglas-rachford-splitting