Adaptive Randomized Averaging Block EBK Method

• Adaptive Randomized Averaging Block Extended Bregman–Kaczmarz (aRABEBK) is an iterative solver for sparse recovery and constrained inverse problems, combining randomized block selection with Bregman projections.
• It employs adaptive relaxation and block averaging to achieve provable linear convergence and enhance robustness against inconsistent or noisy measurements.
• Empirical results demonstrate that aRABEBK reduces iteration counts and CPU time while significantly improving signal recovery compared to classical Kaczmarz methods.

The Adaptive Randomized Averaging Block Extended Bregman–Kaczmarz (aRABEBK) method is a class of iterative solvers targeting sparse recovery, regularized least-squares, and nonlinear inverse problems, where the optimization is constrained by linear (or nonlinear) measurement equations. aRABEBK generalizes single-row Kaczmarz-type methods by combining randomized block selection, averaging of Bregman-projected updates, and a blockwise adaptive stepsize or relaxation parameter. Core features include provable linear convergence rates, parallelization through block updates, robustness to inconsistent/noisy systems, and empirical superiority over classical extended Kaczmarz and nonadaptive Bregman–Kaczmarz approaches (Tondji et al., 2022, Dong et al., 10 Dec 2025, Xiao et al., 21 Feb 2025, Dong et al., 16 Jan 2026).

1. Problem Formulation and Algorithm Structure

The canonical problem addressed by aRABEBK is of the form

$$\min_{x\in\mathbb{R}^n} f(x) \quad \text{s.t.} \quad A\,x = \hat y,$$

where $\hat y = \arg\min_{y\in\mathcal{R}(A)} g^*(b-y)$, $f$ is a strongly convex regularizer (often $\ell_1+\ell_2^2$), and $g^*$ quantifies data-fitting (e.g., squared Euclidean norm). The method operates in the dual space using Bregman distances induced by $f$ and $g^*$.

At iteration $k$, block indices are selected:

• Row blocks $\mathcal{I}_i$ partition measurement equations.
• Column blocks $\mathcal{J}_j$ partition variables.

Two dual-primal pairs $(z_k^*,z_k)$ and $(x_k^*,x_k)$ are maintained, with updates computed per block. The core dual updates (block averaged, extended Bregman–Kaczmarz) are: $$z_{k+1}^* = z_k^* - \alpha_k^{(z)} \|A_{:,\mathcal{J}_{j_k}}\|_F^{-2}\; A_{:,\mathcal{J}_{j_k}} (A_{:,\mathcal{J}_{j_k}}^\top z_k),$$

$$x_{k+1}^* = x_k^* - \alpha_k^{(x)} \|A_{\mathcal{I}_{i_k},:}\|_F^{-2}\; A_{\mathcal{I}_{i_k},:}^\top r_k,$$

where relaxation parameters $\alpha_k^{(z)}, \alpha_k^{(x)}$ are adaptively chosen and $r_k$ is the blockwise residual (Dong et al., 10 Dec 2025, Dong et al., 16 Jan 2026). Primal variables are recovered via Fenchel conjugate mappings (e.g., soft-shrinkage for $\ell_1+\ell_2^2$).

2. Derivation and Theoretical Foundations

aRABEBK can be interpreted via stochastic mirror descent, dual block coordinate descent, and as an averaged extension of the linearized Bregman algorithm. The derivation fuses randomized block selection, blockwise averaging, and Bregman projections in dual space. Key features include:

• Use of Bregman distances $D_f^{x^*}(x,y)$, ensuring strong convexity properties.
• Block adaptive stepsizes determined by local curvature: for block $B$, $$\alpha_B = \mu / (\sigma_{\max}^2(A_B)/\|A_B\|_F^2)$$.
• Averaging schemes across block projections reduce variance and accelerate convergence relative to sequential Kaczmarz updates (Tondji et al., 2022, Xiao et al., 21 Feb 2025).

For nonlinear problems ($F(x)$), block averaging is performed via tangential cone conditions and block Jacobian singular-value bounds, ensuring global geometric decrease in Bregman distance (Xiao et al., 21 Feb 2025).

3. Adaptive Relaxation and Stepsize Selection

The central adaptive component in aRABEBK is the per-iteration relaxation parameter, tuning step magnitude in accordance with block geometry and local residuals. Several parameterizations are used:

• Blockwise curvature/condition number: $\beta_B = \sigma_{\max}^2(A_B)/\|A_B\|_F^2$, and adaptive stepsizes $\alpha_B=\mu/\beta_B$ (Dong et al., 10 Dec 2025).
• Polyak-type residual matching: $\alpha = \frac{\|M r^k\|^2}{\|A^\top M r^k\|_2^2}$ for weighted averaging matrix $M$.
• Empirically, relaxation in $(0,2)$ yields monotonic Bregman reduction.

Heuristic adaptation includes downward adjustment under insufficient decrease or stagnation, and incremental amplification upon favorable progress, while ensuring stability constraints $0<\alpha_k<2\mu/\beta$ (Dong et al., 10 Dec 2025, Dong et al., 16 Jan 2026).

4. Convergence Analysis and Stability

Under appropriate coupling of block probabilities, weights, and relaxation parameters, aRABEBK attains linear convergence in expectation. The core results include:

• Expected contraction of Bregman distance per step:

$$\mathbb{E}[D_f^{x_{k+1}^*}(x_{k+1},\hat x)] \leq (1-c\gamma)\mathbb{E}[D_f^{x_{k}^*}(x_{k},\hat x)] + O(\text{auxiliary error}),$$

with explicit contraction factors based on block singular values and curvature (Dong et al., 10 Dec 2025, Dong et al., 16 Jan 2026).

• Robustness for noisy/inconsistent right-hand sides: iterates converge to a neighborhood of optimal solution, with error proportional to data noise (Tondji et al., 2022).

Block averaging and adaptation both sharpen contraction constants, giving measurable speedups versus nonadaptive or sequential-row Kaczmarz variants (Tondji et al., 2022, Dong et al., 10 Dec 2025, Xiao et al., 21 Feb 2025).

5. Computational Complexity and Implementation

The per-iteration cost is governed by block size $\tau$, with typical complexity $O(\tau (m+n))$ for matrix-vector products and small vector updates in dual spaces. Parallelization across blocks is feasible and crucial for achieving practical wall-clock speedup. Empirical guidelines for block size selection suggest moderate values ($10$-$50$) to balance per-iteration cost and faster per-step contraction.

Recommended stopping criteria include:

• Primal-dual residual threshold ($\|A x^k - b\| < \text{tol}$),
• Relative change ($\|x^{k+1}-x^k\|/\|x^k\|$),
• Maximal iteration/time budgets.

Theoretical iteration complexity shows reduction factors up to $(\|A\|_F^2/\sigma_{\max}^2(A))$ compared to standard Kaczmarz, with diminishing returns for very large block sizes (Tondji et al., 2022, Dong et al., 16 Jan 2026).

6. Numerical Performance and Benchmark Results

Experiments across synthetic Gaussian systems, structured rank-deficient matrices, and large-scale MNIST image-recovery tasks demonstrate:

• 2–20× reduction in iteration count and CPU time for aRABEBK compared to classical (single-row) extended Bregman-Kaczmarz and nonadaptive block methods.
• Improved robustness in ill-conditioned and noisy settings.
• Superior signal recovery quality (e.g., PSNR of $46.35$ for block adaptive vs $13.25$ for nonadaptive Kaczmarz on MNIST sparse LS at $m=500, n=784$) (Dong et al., 16 Jan 2026).
• Consistent empirical linear error decay, with soft-thresholding (Bregman update) critical for sparse recovery accuracy (Dong et al., 10 Dec 2025, Xiao et al., 21 Feb 2025).

A summary table is provided for reference:

Method	Iterations (Sparse LS)	CPU Time (s)	PSNR (MNIST)
REBK	55,189	2.30	13.25
cRABEBK	4,402	0.92	22.59
aRABEBK	2,844	0.39	46.35

Empirical evidence confirms the theoretical advantages of block averaging and adaptive relaxation.

7. Extensions, Nonlinear and Combined Optimization Settings

Nonlinear extensions employ blockwise residual selection based on maximum-residual thresholds, weighted averaging of blockwise Bregman projections, and generalized adaptive stepsize formulas tailored to local Jacobians and tangential cone conditions (Xiao et al., 21 Feb 2025). The method interfaces seamlessly with combined optimization problems that couple data-fidelity, structural constraints, and regularization, generalizing beyond pure least-squares recovery to broad classes of inverse problems (Dong et al., 10 Dec 2025, Dong et al., 16 Jan 2026).

The method integrates, generalizes, and subsumes a variety of earlier frameworks: randomized sparse Kaczmarz, block Kaczmarz, linearized Bregman, and extended Bregman-Kaczmarz, providing a flexible paradigm for efficient and robust reconstruction in high-dimensional, noisy, and structurally heterogeneous regimes.