Randomised Block Coordinate DR Splitting

• The paper introduces a DR splitting framework that activates random block subsets per iteration for solving monotone inclusions in product Hilbert spaces.
• It extends classic Douglas-Rachford methods to handle inexact proximal evaluations and stochastic perturbations, enabling applications in logistic regression and federated optimization.
• The approach achieves convergence guarantees ranging from almost-sure weak convergence to linear rates, with practical acceleration via heavy-ball momentum and low per-iteration cost.

Searching arXiv for the cited papers and closely related randomized/block-coordinate Douglas–Rachford work. Randomised block coordinate Douglas-Rachford splitting denotes a family of Douglas-Rachford (DR) schemes in which only a random subset of blocks is activated at each iteration, typically in a product Hilbert space and often under stochastic perturbations, inexact proximal evaluations, or asynchronous execution. In the operator-theoretic formulation, the method addresses monotone inclusions of the form $0\in A x + B x$ with block structure and i.i.d. random sweeping rules, yielding almost-sure convergence results in Hilbert spaces (Combettes et al., 2014). Subsequent developments specialized the same randomized partial-update principle to sparse binary logistic regression through random block-coordinate DR splitting with blockwise preconditioning and a closed-form proximity operator for the logistic loss (Briceno-Arias et al., 2017), to nonconvex federated composite optimization through FedDR and asyncFedDR (Tran-Dinh et al., 2021), and to linear feasibility problems through a randomized $r$-sets DR variant with linear convergence in expectation and heavy-ball acceleration (Han et al., 2022).

1. Foundational formulation

The foundational setting is the product Hilbert space

$$\mathcal H=\mathcal H_1\oplus\mathcal H_2\oplus\cdots\oplus\mathcal H_m,$$

equipped with two maximally monotone operators

$$A\colon \mathcal H\to 2^{\mathcal H}, \qquad B\colon \mathcal H\to 2^{\mathcal H},$$

with block structure

$$A(x_1,\dots,x_m)= (A_1 x_1,\dots,A_m x_m),\qquad B(x_1,\dots,x_m)= (B_1(x_1,\dots,x_m),\dots,B_m(x_1,\dots,x_m)).$$

The associated primal inclusion is

$$\text{find }x=(x_1,\dots,x_m)\in\mathcal H\quad\text{such that}\quad 0\in A x + B x,$$

with solution set $\mathsf F=\mathrm{zer}(A+B)\neq\varnothing$ (Combettes et al., 2014).

The block-coordinate DR viewpoint relies on the resolvents

$$J_{\gamma A}=(\mathrm{Id}+\gamma A)^{-1}, \qquad J_{\gamma B}=(\mathrm{Id}+\gamma B)^{-1},$$

where $J_{\gamma A}$ splits blockwise,

$$J_{\gamma A}(x_1,\dots,x_m)=\bigl(J_{\gamma A_1}x_1,\dots,J_{\gamma A_m}x_m\bigr),$$

and $r$0 defines the component maps $r$1. This formulation places randomized block activation inside the classical DR resolvent-reflection architecture rather than treating it as an external stochastic wrapper.

Paper	Problem class	Distinctive randomized feature
(Combettes et al., 2014)	$r$2 in product Hilbert spaces	random sweeping rules to select arbitrarily the blocks
(Briceno-Arias et al., 2017)	sparse binary logistic regression	randomly selecting a mini-batch of data and update the variables in a block coordinate manner
(Tran-Dinh et al., 2021)	nonconvex federated composite optimization	update only a subset of users at each communication round
(Han et al., 2022)	feasibility problem derived from linear systems	randomized $r$3-sets-DR and heavy ball momentum

This family should therefore be understood as an operator-splitting framework whose randomized structure is expressed at the level of activated coordinates, samples, users, or reflected sets, depending on the application.

2. Random sweeping, block updates, and asymptotic convergence

In the general block-coordinate construction, iteration $r$4 activates a random subset of the $r$5 blocks through an i.i.d. binary vector

$r$6

with

$r$7

and $r$8 independent of the past. Additive stochastic errors $r$9 are permitted under square-integrability conditions. Given $$\mathcal H=\mathcal H_1\oplus\mathcal H_2\oplus\cdots\oplus\mathcal H_m,$$0 and relaxation parameters $$\mathcal H=\mathcal H_1\oplus\mathcal H_2\oplus\cdots\oplus\mathcal H_m,$$1 with $$\mathcal H=\mathcal H_1\oplus\mathcal H_2\oplus\cdots\oplus\mathcal H_m,$$2 and $$\mathcal H=\mathcal H_1\oplus\mathcal H_2\oplus\cdots\oplus\mathcal H_m,$$3, the block-coordinate DR iteration is (Combettes et al., 2014)

$$\mathcal H=\mathcal H_1\oplus\mathcal H_2\oplus\cdots\oplus\mathcal H_m,$$4

$$\mathcal H=\mathcal H_1\oplus\mathcal H_2\oplus\cdots\oplus\mathcal H_m,$$5

$$\mathcal H=\mathcal H_1\oplus\mathcal H_2\oplus\cdots\oplus\mathcal H_m,$$6

$$\mathcal H=\mathcal H_1\oplus\mathcal H_2\oplus\cdots\oplus\mathcal H_m,$$7

The corresponding convergence theorem establishes that, if $$\mathcal H=\mathcal H_1\oplus\mathcal H_2\oplus\cdots\oplus\mathcal H_m,$$8, if the sweeping vectors are i.i.d. with positive activation probabilities and independent of the past, if the error moments are finite, and if the relaxation parameters remain in a compact subset of $$\mathcal H=\mathcal H_1\oplus\mathcal H_2\oplus\cdots\oplus\mathcal H_m,$$9, then $$A\colon \mathcal H\to 2^{\mathcal H}, \qquad B\colon \mathcal H\to 2^{\mathcal H},$$0 converges weakly almost surely to an $$A\colon \mathcal H\to 2^{\mathcal H}, \qquad B\colon \mathcal H\to 2^{\mathcal H},$$1-valued random variable. Under the additional assumptions that $$A\colon \mathcal H\to 2^{\mathcal H}, \qquad B\colon \mathcal H\to 2^{\mathcal H},$$2 is weakly sequentially continuous and $$A\colon \mathcal H\to 2^{\mathcal H}, \qquad B\colon \mathcal H\to 2^{\mathcal H},$$3 almost surely, the auxiliary sequence $$A\colon \mathcal H\to 2^{\mathcal H}, \qquad B\colon \mathcal H\to 2^{\mathcal H},$$4 converges strongly almost surely to $$A\colon \mathcal H\to 2^{\mathcal H}, \qquad B\colon \mathcal H\to 2^{\mathcal H},$$5, and

$$A\colon \mathcal H\to 2^{\mathcal H}, \qquad B\colon \mathcal H\to 2^{\mathcal H},$$6

strongly almost surely, where $$A\colon \mathcal H\to 2^{\mathcal H}, \qquad B\colon \mathcal H\to 2^{\mathcal H},$$7 lies in the set of dual solutions (Combettes et al., 2014).

No explicit rate is given in that theorem. The significance of the result lies in the stochastic quasi-Fejér framework: random block activation and stochastic perturbations are incorporated directly into a Krasnoselʹskiĭ-Mann analysis of the DR operator composition. When $$A\colon \mathcal H\to 2^{\mathcal H}, \qquad B\colon \mathcal H\to 2^{\mathcal H},$$8, the scheme reduces exactly to the standard relaxed DR iteration with stochastic errors,

$$A\colon \mathcal H\to 2^{\mathcal H}, \qquad B\colon \mathcal H\to 2^{\mathcal H},$$9

so the block-coordinate theory contains the classical single-block case as a special instance.

3. Sparse binary logistic regression and low-complexity block preconditioning

A concrete specialization appears in sparse logistic regression, where a stochastic version of DR is used to sweep the training set by randomly selecting a mini-batch of data at each iteration and to update variables in a block coordinate manner (Briceno-Arias et al., 2017). The underlying composite convex problem is posed in product spaces: $$A(x_1,\dots,x_m)= (A_1 x_1,\dots,A_m x_m),\qquad B(x_1,\dots,x_m)= (B_1(x_1,\dots,x_m),\dots,B_m(x_1,\dots,x_m)).$$0 where $$A(x_1,\dots,x_m)= (A_1 x_1,\dots,A_m x_m),\qquad B(x_1,\dots,x_m)= (B_1(x_1,\dots,x_m),\dots,B_m(x_1,\dots,x_m)).$$1, each $$A(x_1,\dots,x_m)= (A_1 x_1,\dots,A_m x_m),\qquad B(x_1,\dots,x_m)= (B_1(x_1,\dots,x_m),\dots,B_m(x_1,\dots,x_m)).$$2 is convex with $$A(x_1,\dots,x_m)= (A_1 x_1,\dots,A_m x_m),\qquad B(x_1,\dots,x_m)= (B_1(x_1,\dots,x_m),\dots,B_m(x_1,\dots,x_m)).$$3-Lipschitz gradient, and $$A(x_1,\dots,x_m)= (A_1 x_1,\dots,A_m x_m),\qquad B(x_1,\dots,x_m)= (B_1(x_1,\dots,x_m),\dots,B_m(x_1,\dots,x_m)).$$4 are bounded linear operators. Equivalently, the problem can be written as the primal inclusion

$$A(x_1,\dots,x_m)= (A_1 x_1,\dots,A_m x_m),\qquad B(x_1,\dots,x_m)= (B_1(x_1,\dots,x_m),\dots,B_m(x_1,\dots,x_m)).$$5

and standard DR splitting is then applied to the maximal-monotone operators $$A(x_1,\dots,x_m)= (A_1 x_1,\dots,A_m x_m),\qquad B(x_1,\dots,x_m)= (B_1(x_1,\dots,x_m),\dots,B_m(x_1,\dots,x_m)).$$6 and $$A(x_1,\dots,x_m)= (A_1 x_1,\dots,A_m x_m),\qquad B(x_1,\dots,x_m)= (B_1(x_1,\dots,x_m),\dots,B_m(x_1,\dots,x_m)).$$7.

The random block-coordinate stochastic adaptation introduces a mask

$$A(x_1,\dots,x_m)= (A_1 x_1,\dots,A_m x_m),\qquad B(x_1,\dots,x_m)= (B_1(x_1,\dots,x_m),\dots,B_m(x_1,\dots,x_m)).$$8

with i.i.d. components independent of the past and positive activation probability for every primal and dual block. Possible numerical errors in proximal computations are represented by additive terms $$A(x_1,\dots,x_m)= (A_1 x_1,\dots,A_m x_m),\qquad B(x_1,\dots,x_m)= (B_1(x_1,\dots,x_m),\dots,B_m(x_1,\dots,x_m)).$$9 and $$\text{find }x=(x_1,\dots,x_m)\in\mathcal H\quad\text{such that}\quad 0\in A x + B x,$$0 satisfying summable-error conditions. The resulting RBC-DRS scheme uses blockwise preconditioners

$$\text{find }x=(x_1,\dots,x_m)\in\mathcal H\quad\text{such that}\quad 0\in A x + B x,$$1

together with auxiliary quantities

$$\text{find }x=(x_1,\dots,x_m)\in\mathcal H\quad\text{such that}\quad 0\in A x + B x,$$2

and relaxation $$\text{find }x=(x_1,\dots,x_m)\in\mathcal H\quad\text{such that}\quad 0\in A x + B x,$$3.

For binary logistic regression, the sample loss is

$$\text{find }x=(x_1,\dots,x_m)\in\mathcal H\quad\text{such that}\quad 0\in A x + B x,$$4

with derivative

$$\text{find }x=(x_1,\dots,x_m)\in\mathcal H\quad\text{such that}\quad 0\in A x + B x,$$5

and Lipschitz constant $$\text{find }x=(x_1,\dots,x_m)\in\mathcal H\quad\text{such that}\quad 0\in A x + B x,$$6. Its proximity operator admits the closed form

$$\text{find }x=(x_1,\dots,x_m)\in\mathcal H\quad\text{such that}\quad 0\in A x + B x,$$7

where $$\text{find }x=(x_1,\dots,x_m)\in\mathcal H\quad\text{such that}\quad 0\in A x + B x,$$8 is the generalized Lambert $$\text{find }x=(x_1,\dots,x_m)\in\mathcal H\quad\text{such that}\quad 0\in A x + B x,$$9 function solving

$\mathsf F=\mathrm{zer}(A+B)\neq\varnothing$0

and the branch chosen is the unique nonnegative, increasing one (Briceno-Arias et al., 2017). As $\mathsf F=\mathrm{zer}(A+B)\neq\varnothing$1, the expansion

$\mathsf F=\mathrm{zer}(A+B)\neq\varnothing$2

is used to avoid overflow in practice.

The convergence statement for RBC-DRS is almost-sure convergence: if the masks are i.i.d., independent of the states, each block has positive activation probability, the deterministic errors satisfy the stated summability conditions, and the step parameters and relaxation are chosen as required, then the sequence $\mathsf F=\mathrm{zer}(A+B)\neq\varnothing$3 converges almost surely, weakly in each $\mathsf F=\mathrm{zer}(A+B)\neq\varnothing$4, to a solution of the original problem. The low-complexity emphasis is explicit in the per-iteration accounting: the inverses $\mathsf F=\mathrm{zer}(A+B)\neq\varnothing$5 are precomputed once with cost $\mathsf F=\mathrm{zer}(A+B)\neq\varnothing$6 if $\mathsf F=\mathrm{zer}(A+B)\neq\varnothing$7; each activated primal block requires one multiplication $\mathsf F=\mathrm{zer}(A+B)\neq\varnothing$8 with cost $\mathsf F=\mathrm{zer}(A+B)\neq\varnothing$9 and one prox of $$J_{\gamma A}=(\mathrm{Id}+\gamma A)^{-1}, \qquad J_{\gamma B}=(\mathrm{Id}+\gamma B)^{-1},$$0; each activated dual block uses a scalar prox of $$J_{\gamma A}=(\mathrm{Id}+\gamma A)^{-1}, \qquad J_{\gamma B}=(\mathrm{Id}+\gamma B)^{-1},$$1; and if on average a fraction $$J_{\gamma A}=(\mathrm{Id}+\gamma A)^{-1}, \qquad J_{\gamma B}=(\mathrm{Id}+\gamma B)^{-1},$$2 of primal blocks and $$J_{\gamma A}=(\mathrm{Id}+\gamma A)^{-1}, \qquad J_{\gamma B}=(\mathrm{Id}+\gamma B)^{-1},$$3 of dual blocks is activated, the expected per-iteration cost is

$$J_{\gamma A}=(\mathrm{Id}+\gamma A)^{-1}, \qquad J_{\gamma B}=(\mathrm{Id}+\gamma B)^{-1},$$4

The paper’s experiments on standard datasets report efficiency with respect to stochastic gradient-like methods.

4. Federated composite optimization: FedDR and asyncFedDR

In federated learning, the same design principle is realized as randomized user activation in a nonconvex composite problem,

$$J_{\gamma A}=(\mathrm{Id}+\gamma A)^{-1}, \qquad J_{\gamma B}=(\mathrm{Id}+\gamma B)^{-1},$$5

where each $$J_{\gamma A}=(\mathrm{Id}+\gamma A)^{-1}, \qquad J_{\gamma B}=(\mathrm{Id}+\gamma B)^{-1},$$6 is $$J_{\gamma A}=(\mathrm{Id}+\gamma A)^{-1}, \qquad J_{\gamma B}=(\mathrm{Id}+\gamma B)^{-1},$$7 and $$J_{\gamma A}=(\mathrm{Id}+\gamma A)^{-1}, \qquad J_{\gamma B}=(\mathrm{Id}+\gamma B)^{-1},$$8-smooth, but not necessarily convex, and $$J_{\gamma A}=(\mathrm{Id}+\gamma A)^{-1}, \qquad J_{\gamma B}=(\mathrm{Id}+\gamma B)^{-1},$$9 is proper, closed, convex and proximable. The target is an $J_{\gamma A}$0-stationary point satisfying

$J_{\gamma A}$1

At each round $J_{\gamma A}$2, only a random subset $J_{\gamma A}$3 of active clients is chosen, with

$J_{\gamma A}$4

independently of past updates; in practice $J_{\gamma A}$5 can be uniform sampling of $J_{\gamma A}$6 clients out of $J_{\gamma A}$7, non-uniform importance sampling, or full participation (Tran-Dinh et al., 2021).

The synchronous FedDR algorithm maintains per-client variables $J_{\gamma A}$8 and server variables $J_{\gamma A}$9. For active clients,

$$J_{\gamma A}(x_1,\dots,x_m)=\bigl(J_{\gamma A_1}x_1,\dots,J_{\gamma A_m}x_m\bigr),$$0

$$J_{\gamma A}(x_1,\dots,x_m)=\bigl(J_{\gamma A_1}x_1,\dots,J_{\gamma A_m}x_m\bigr),$$1

$$J_{\gamma A}(x_1,\dots,x_m)=\bigl(J_{\gamma A_1}x_1,\dots,J_{\gamma A_m}x_m\bigr),$$2

while the server performs

$$J_{\gamma A}(x_1,\dots,x_m)=\bigl(J_{\gamma A_1}x_1,\dots,J_{\gamma A_m}x_m\bigr),$$3

In compact form, the algorithm is a block-coordinate variant of DR splitting applied to the constrained reformulation $$J_{\gamma A}(x_1,\dots,x_m)=\bigl(J_{\gamma A_1}x_1,\dots,J_{\gamma A_m}x_m\bigr),$$4 with regularizer $$J_{\gamma A}(x_1,\dots,x_m)=\bigl(J_{\gamma A_1}x_1,\dots,J_{\gamma A_m}x_m\bigr),$$5 on $$J_{\gamma A}(x_1,\dots,x_m)=\bigl(J_{\gamma A_1}x_1,\dots,J_{\gamma A_m}x_m\bigr),$$6. The asynchronous variant, asyncFedDR, lets exactly one client wake up at iteration $$J_{\gamma A}(x_1,\dots,x_m)=\bigl(J_{\gamma A_1}x_1,\dots,J_{\gamma A_m}x_m\bigr),$$7, read a possibly stale model $$J_{\gamma A}(x_1,\dots,x_m)=\bigl(J_{\gamma A_1}x_1,\dots,J_{\gamma A_m}x_m\bigr),$$8 with delay bounded by $$J_{\gamma A}(x_1,\dots,x_m)=\bigl(J_{\gamma A_1}x_1,\dots,J_{\gamma A_m}x_m\bigr),$$9, perform the same three local steps, and send $r$00 to the server, which immediately updates $r$01 and $r$02.

The main complexity statements are explicit. Under Assumptions A1–A3 and sufficiently small $r$03, Theorem 3.1 gives

$r$04

With exact prox, $r$05, and $r$06, one obtains

$r$07

hence $r$08 rounds to reach $r$09. Under A1, A2, A4, and suitable $r$10, Theorem 3.2 gives for asyncFedDR

$r$11

hence also $r$12 communication complexity. Both theorems are stated to match the known lower bound up to constants even in the presence of inexact prox’s and asynchrony.

The empirical evidence covers synthetic quadratic-plus-$r$13 problems, FEMNIST, composite $r$14 regularization, and asynchronous MNIST experiments with simulated client delays. The reported findings are that FedDR and FedPD outperform FedProx and FedAvg on synthetic problems, especially in highly non-iid cases; on FEMNIST, FedDR reaches lower loss and higher test-accuracy per byte of communication than FedPD/FedProx/FedAvg; composite $r$15 experiments confirm robustness under inexact prox implemented by local SGD; and asyncFedDR runs up to $r$16 faster in wall-clock time than synchronous FedDR while preserving convergence speed in rounds.

5. Linear feasibility, randomized $r$17-sets DR, and heavy-ball acceleration

A related randomized DR construction for linear systems begins from the consistent problem

$r$18

which is rewritten as the convex feasibility problem

$r$19

where $r$20 is the $r$21th row of $r$22 and $r$23 the $r$24th entry of $r$25. The orthogonal projector and reflection are

$r$26

(Han et al., 2022).

The randomized $r$27-sets Douglas-Rachford method (RrDR), with relaxation parameter $r$28 and block-size $r$29, initializes $r$30 and for $r$31 samples

$r$32

then applies the reflections

$r$33

followed by the relaxed update

$r$34

For

$r$35

the projection of $r$36 onto the solution set of $r$37, and for $r$38 the smallest nonzero singular value of $r$39, the main theorem states

$r$40

The contraction factor depends only on the singular-value ratio $r$41 and the block-size $r$42, but not on the ambient dimension $r$43. This is significant because the direct extension of DR to more-than-two-sets feasibility, called the $r$44-sets-DR method, is stated not to be necessarily convergent, whereas the randomized version brings linear convergence in expectation.

The momentum extension mRrDR adds Polyak heavy-ball momentum,

$r$45

If $r$46 and $r$47 satisfy $r$48, then for $r$49,

$r$50

where

$r$51

For the norm of the expected iterate, under mild conditions,

$r$52

which is described as linear decay at rate $r$53, achieving a square-root speedup over the relaxation-only bound. The numerical experiments include synthetic Gaussian systems, SuiteSparse and LIBSVM datasets such as Franz1 and ijcnn1, and randomized gossip or average-consensus systems on cycle, path, and random geometric graphs. The reported empirical findings include faster convergence than randomized Kaczmarz when $r$54, significant acceleration for mRrDR with $r$55, often $r$56–$r$57 higher efficiency than RK, RGS, and RP-ADMM in row-action count and CPU time, and superiority over MATLAB’s dense-solver routines for large overdetermined consistent systems once $r$58.

6. Scope, recurring themes, and common misconceptions

Several recurring themes cut across these works. First, randomised block coordinate DR is not restricted to exact deterministic proximal computation. The Hilbert-space framework explicitly permits additive stochastic errors in the evaluation of the resolvents (Combettes et al., 2014). The logistic-regression variant models additive errors $r$59 and $r$60 under summable-error conditions (Briceno-Arias et al., 2017). FedDR allows inexact local solves through approximate evaluations of $r$61, and asyncFedDR additionally tolerates bounded staleness without locking (Tran-Dinh et al., 2021). A common misconception is therefore that DR-based splitting requires exact full-batch proximal subproblems at every step; the randomized block-coordinate literature shows otherwise.

Second, the framework is not limited to convex finite-dimensional regression. The 2014 block-coordinate theory is formulated for structured monotone inclusion and convex minimization problems in Hilbert spaces (Combettes et al., 2014). The 2017 RBC-DRS instantiation targets sparse logistic regression (Briceno-Arias et al., 2017). FedDR and asyncFedDR treat nonconvex federated composite optimization with convex regularizers (Tran-Dinh et al., 2021). The randomized $r$62-sets DR method addresses linear feasibility induced by linear systems and augments it with heavy-ball momentum (Han et al., 2022). This suggests that randomized partial activation is best viewed as a structural modification of the DR iteration rather than as an application-specific heuristic.

Third, convergence guarantees depend strongly on the problem class. In the general block-coordinate monotone setting, the available guarantees are weak almost-sure convergence of $r$63 and, under additional continuity and vanishing-error assumptions, strong almost-sure convergence of $r$64, with no explicit rate (Combettes et al., 2014). In federated nonconvex optimization, the relevant guarantee is an $r$65 bound on the average squared gradient mapping norm, hence $r$66 communication complexity (Tran-Dinh et al., 2021). In linear systems, randomization yields linear convergence in expectation, and heavy-ball momentum yields an accelerated rate for the norm of the expected error (Han et al., 2022). A second common misconception is therefore that all randomized DR variants admit a single universal rate statement; the theory is in fact highly regime-dependent.

Finally, relation to the classical DR method remains direct rather than merely analogical. When $r$67, the block-coordinate Hilbert-space scheme reduces exactly to the standard relaxed DR iteration with stochastic errors (Combettes et al., 2014). The logistic-regression method remains a DR splitting on maximal-monotone operators with random block and mini-batch activation (Briceno-Arias et al., 2017). FedDR is stated compactly as a block-coordinate variant of DR splitting for the constrained reformulation $r$68 (Tran-Dinh et al., 2021). The linear-system RrDR preserves the projector-reflector core of DR while randomizing reflected hyperplanes over $r$69 successive draws (Han et al., 2022). Taken together, these results indicate that randomised block coordinate Douglas-Rachford splitting is a coherent extension of DR methodology in which partial activation, stochastic approximation, and structural decomposition are combined without abandoning the underlying resolvent-reflection geometry.