---
title: Adaptive Randomized Averaging Block EBK Method
url: https://www.emergentmind.com/topics/adaptive-randomized-averaging-block-extended-bregman-kaczmarz-arabebk-method
type: topic
---

# Adaptive Randomized Averaging Block EBK Method

The Adaptive Randomized Averaging Block Extended Bregman–Kaczmarz (aRABEBK) method is a class of iterative solvers targeting sparse recovery, regularized least-squares, and nonlinear inverse problems, where the optimization is constrained by linear (or nonlinear) measurement equations. aRABEBK generalizes single-row Kaczmarz-type methods by combining randomized block selection, averaging of Bregman-projected updates, and a blockwise adaptive stepsize or relaxation parameter. Core features include provable linear convergence rates, parallelization through block updates, robustness to inconsistent/noisy systems, and empirical superiority over classical extended Kaczmarz and nonadaptive Bregman–Kaczmarz approaches [2203.10838], [2512.09825], [2502.15529], [2601.11157].

## 1. Problem Formulation and Algorithm Structure

The canonical problem addressed by aRABEBK is of the form
\[
\min_{x\in\mathbb{R}^n} f(x) \quad \text{s.t.} \quad A\,x = \hat y,
\]
where \(\hat y = \arg\min_{y\in\mathcal{R}(A)} g^*(b-y)\), \(f\) is a strongly convex regularizer (often \(\ell_1+\ell_2^2\)), and \(g^*\) quantifies data-fitting (e.g., squared Euclidean norm). The method operates in the dual space using Bregman distances induced by \(f\) and \(g^*\).

At iteration \(k\), block indices are selected:
- Row blocks \(\mathcal{I}_i\) partition measurement equations.
- Column blocks \(\mathcal{J}_j\) partition variables.

Two dual-primal pairs \((z_k^*,z_k)\) and \((x_k^*,x_k)\) are maintained, with updates computed per block. The core dual updates (block averaged, extended Bregman–Kaczmarz) are:
\[
z_{k+1}^* = z_k^* - \alpha_k^{(z)} \|A_{:,\mathcal{J}_{j_k}}\|_F^{-2}\; A_{:,\mathcal{J}_{j_k}} (A_{:,\mathcal{J}_{j_k}}^\top z_k),
\]
\[
x_{k+1}^* = x_k^* - \alpha_k^{(x)} \|A_{\mathcal{I}_{i_k},:}\|_F^{-2}\; A_{\mathcal{I}_{i_k},:}^\top r_k,
\]
where relaxation parameters \(\alpha_k^{(z)}, \alpha_k^{(x)}\) are adaptively chosen and \(r_k\) is the blockwise residual [2512.09825], [2601.11157]. Primal variables are recovered via Fenchel conjugate mappings (e.g., soft-shrinkage for \(\ell_1+\ell_2^2\)).

## 2. Derivation and Theoretical Foundations

aRABEBK can be interpreted via stochastic mirror descent, dual block coordinate descent, and as an averaged extension of the linearized Bregman algorithm. The derivation fuses randomized block selection, blockwise averaging, and Bregman projections in dual space. Key features include:
- Use of Bregman distances \(D_f^{x^*}(x,y)\), ensuring strong convexity properties.
- Block adaptive stepsizes determined by local curvature: for block \(B\), \(\alpha_B = \mu / (\sigma_{\max}^2(A_B)/\|A_B\|_F^2)\).
- Averaging schemes across block projections reduce variance and accelerate convergence relative to sequential Kaczmarz updates [2203.10838], [2502.15529].

For nonlinear problems (\(F(x)\)), block averaging is performed via tangential cone conditions and block Jacobian singular-value bounds, ensuring global geometric decrease in Bregman distance [2502.15529].

## 3. Adaptive Relaxation and Stepsize Selection

The central adaptive component in aRABEBK is the per-iteration relaxation parameter, tuning step magnitude in accordance with block geometry and local residuals. Several parameterizations are used:
- Blockwise curvature/condition number: \(\beta_B = \sigma_{\max}^2(A_B)/\|A_B\|_F^2\), and adaptive stepsizes \(\alpha_B=\mu/\beta_B\) [2512.09825].
- Polyak-type residual matching: \(\alpha = \frac{\|M r^k\|^2}{\|A^\top M r^k\|_2^2}\) for weighted averaging matrix \(M\).
- Empirically, relaxation in \((0,2)\) yields monotonic Bregman reduction.

Heuristic adaptation includes downward adjustment under insufficient decrease or stagnation, and incremental amplification upon favorable progress, while ensuring stability constraints \(0<\alpha_k<2\mu/\beta\) [2512.09825], [2601.11157].

## 4. Convergence Analysis and Stability

Under appropriate coupling of block probabilities, weights, and relaxation parameters, aRABEBK attains linear convergence in expectation. The core results include:
- Expected contraction of Bregman distance per step:
  \[
  \mathbb{E}[D_f^{x_{k+1}^*}(x_{k+1},\hat x)] \leq (1-c\gamma)\mathbb{E}[D_f^{x_{k}^*}(x_{k},\hat x)] + O(\text{auxiliary error}),
  \]
  with explicit contraction factors based on block singular values and curvature [2512.09825], [2601.11157].
- Robustness for noisy/inconsistent right-hand sides: iterates converge to a neighborhood of optimal solution, with error proportional to data noise [2203.10838].

Block averaging and adaptation both sharpen contraction constants, giving measurable speedups versus nonadaptive or sequential-row Kaczmarz variants [2203.10838], [2512.09825], [2502.15529].

## 5. Computational Complexity and Implementation

The per-iteration cost is governed by block size \(\tau\), with typical complexity \(O(\tau (m+n))\) for matrix-vector products and small vector updates in dual spaces. Parallelization across blocks is feasible and crucial for achieving practical wall-clock speedup. Empirical guidelines for block size selection suggest moderate values (\(10\)-\(50\)) to balance per-iteration cost and faster per-step contraction.

Recommended stopping criteria include:
- Primal-dual residual threshold (\(\|A x^k - b\| < \text{tol}\)),
- Relative change (\(\|x^{k+1}-x^k\|/\|x^k\|\)),
- Maximal iteration/time budgets.

Theoretical iteration complexity shows reduction factors up to \((\|A\|_F^2/\sigma_{\max}^2(A))\) compared to standard Kaczmarz, with diminishing returns for very large block sizes [2203.10838], [2601.11157].

## 6. Numerical Performance and Benchmark Results

Experiments across synthetic Gaussian systems, structured rank-deficient matrices, and large-scale MNIST image-recovery tasks demonstrate:
- 2–20× reduction in iteration count and CPU time for aRABEBK compared to classical (single-row) extended Bregman-Kaczmarz and nonadaptive block methods.
- Improved robustness in ill-conditioned and noisy settings.
- Superior signal recovery quality (e.g., PSNR of \(46.35\) for block adaptive vs \(13.25\) for nonadaptive Kaczmarz on MNIST sparse LS at \(m=500, n=784\)) [2601.11157].
- Consistent empirical linear error decay, with soft-thresholding (Bregman update) critical for sparse recovery accuracy [2512.09825], [2502.15529].

A summary table is provided for reference:

| Method            | Iterations (Sparse LS) | CPU Time (s) | PSNR (MNIST) |
|-------------------|------------------------|--------------|--------------|
| REBK              | 55,189                 | 2.30         | 13.25        |
| cRABEBK           | 4,402                  | 0.92         | 22.59        |
| aRABEBK           | 2,844                  | 0.39         | 46.35        |

Empirical evidence confirms the theoretical advantages of block averaging and adaptive relaxation.

## 7. Extensions, Nonlinear and Combined Optimization Settings

Nonlinear extensions employ blockwise residual selection based on maximum-residual thresholds, weighted averaging of blockwise Bregman projections, and generalized adaptive stepsize formulas tailored to local Jacobians and tangential cone conditions [2502.15529]. The method interfaces seamlessly with combined optimization problems that couple data-fidelity, structural constraints, and regularization, generalizing beyond pure least-squares recovery to broad classes of inverse problems [2512.09825], [2601.11157].

The method integrates, generalizes, and subsumes a variety of earlier frameworks: randomized sparse Kaczmarz, block Kaczmarz, linearized Bregman, and extended Bregman-Kaczmarz, providing a flexible paradigm for efficient and robust reconstruction in high-dimensional, noisy, and structurally heterogeneous regimes.

Source: https://www.emergentmind.com/topics/adaptive-randomized-averaging-block-extended-bregman-kaczmarz-arabebk-method