Random Greedy Fast Block Kaczmarz

• The paper introduces a pseudoinverse-free randomized block Kaczmarz method that employs a one-dimensional weighted residual update for solving nonlinear systems.
• It combines random sampling with greedy selection to target high-residual equations, reducing computational cost while ensuring linear convergence under a tangential cone condition.
• Empirical tests on large-scale problems, including the Chandrasekhar H-equation and Broyden tridiagonal function, demonstrate its superior efficiency over traditional block methods.

Searching arXiv for the named method and closely related Kaczmarz variants to ground the article in current papers. Random Greedy Fast Block Kaczmarz denotes a class of Kaczmarz-type iterative methods that combine random subsampling, greedy residual-based block selection, and low-cost block updates to solve large-scale systems, especially nonlinear systems. In the 2025 formulation for nonlinear equations, the method is presented for solving $F(x)=0$ with $F:\mathbb{R}^n\to\mathbb{R}^m$, and its defining feature is the replacement of Jacobian-submatrix pseudoinversion by a one-dimensional weighted residual step, yielding a pseudoinverse-free block update with linear convergence in expectation under a local tangential cone condition (Ding et al., 13 Aug 2025). Related linear-system methods share the same structural motif—random sampling followed by greedy choice of high-residual rows—but differ in whether they use pseudoinverse-based projections, averaged updates, or extrapolated stepsizes (Zhang et al., 2022, Xiao et al., 2022, Necoara, 2019).

1. Conceptual setting and problem class

The nonlinear RGFBK method is designed for the system

$F(x)=0,\qquad x\in D\subset\mathbb{R}^n,$

where $F=(F_1,\dots,F_m)^T$ is continuously differentiable on a closed domain $D$, and $F'(x)\in\mathbb{R}^{m\times n}$ denotes the Jacobian (Ding et al., 13 Aug 2025). For an index set $I\subseteq[m]:=\{1,\dots,m\}$, the notation

$$F_I(x)=(F_i(x))_{i\in I},\qquad F'_I(x)\in\mathbb{R}^{|I|\times n}$$

is used to isolate sampled or greedily selected equations (Ding et al., 13 Aug 2025).

The method is motivated by the cost profile of classical block Kaczmarz schemes. In a block formulation, one typically enforces multiple equations simultaneously through a block correction involving the Moore–Penrose pseudoinverse of a Jacobian or coefficient submatrix. For large blocks or high-dimensional problems, that step can dominate the iteration cost. The nonlinear RGFBK method avoids this expense by using only one matrix–vector product of the form $F'_I(x_k)^T\omega_k$, while retaining a block-selection mechanism that targets informative equations (Ding et al., 13 Aug 2025).

This places RGFBK at the intersection of several Kaczmarz lineages. Randomized block Kaczmarz emphasizes stochastic row or block sampling and analyzes convergence through conditioning of sampled submatrices (Necoara, 2019). Greedy and Motzkin-type variants prioritize equations with large residuals or violations (Zhang et al., 2020). Fast greedy block methods remove explicit pseudoinversion in the linear setting by replacing orthogonal block projections with averaged Kaczmarz-type updates (Xiao et al., 2022). Nonlinear greedy randomized sampling methods extend these ideas to Jacobian-based linearizations of $f(x)=0$ under a local tangential cone condition (Zhang et al., 2022). The 2025 nonlinear RGFBK paper integrates these strands into a single pseudoinverse-free block method (Ding et al., 13 Aug 2025).

2. Algorithmic structure

The nonlinear RGFBK algorithm takes as inputs an initial guess $F:\mathbb{R}^n\to\mathbb{R}^m$0, a control-sample size $F:\mathbb{R}^n\to\mathbb{R}^m$1, a greedy-block size $F:\mathbb{R}^n\to\mathbb{R}^m$2, and a relaxation parameter $F:\mathbb{R}^n\to\mathbb{R}^m$3 (Ding et al., 13 Aug 2025). At iteration $F:\mathbb{R}^n\to\mathbb{R}^m$4, it first samples uniformly without replacement $F:\mathbb{R}^n\to\mathbb{R}^m$5 indices $F:\mathbb{R}^n\to\mathbb{R}^m$6. It then computes the residual magnitudes $F:\mathbb{R}^n\to\mathbb{R}^m$7 and retains the $F:\mathbb{R}^n\to\mathbb{R}^m$8 indices in $F:\mathbb{R}^n\to\mathbb{R}^m$9 with largest $F(x)=0,\qquad x\in D\subset\mathbb{R}^n,$0; this greedy subset is denoted $F(x)=0,\qquad x\in D\subset\mathbb{R}^n,$1 (Ding et al., 13 Aug 2025).

The selected residual block becomes the weight vector,

$F(x)=0,\qquad x\in D\subset\mathbb{R}^n,$2

and the update direction is

$F(x)=0,\qquad x\in D\subset\mathbb{R}^n,$3

The next iterate is computed as

$F(x)=0,\qquad x\in D\subset\mathbb{R}^n,$4

The iteration terminates when $F(x)=0,\qquad x\in D\subset\mathbb{R}^n,$5 falls below tolerance or a maximum iteration count is reached (Ding et al., 13 Aug 2025).

Two aspects are structurally central. First, the method is “random greedy”: random sampling limits the search space from all $F(x)=0,\qquad x\in D\subset\mathbb{R}^n,$6 equations to a manageable subset of size $F(x)=0,\qquad x\in D\subset\mathbb{R}^n,$7, and greedy filtering concentrates the update on the largest residuals within that subset (Ding et al., 13 Aug 2025). Second, the update is “fast” in the specific sense used in prior fast greedy block Kaczmarz work: it avoids solving a block least-squares problem via pseudoinverse and instead performs a directionally scaled residual step (Xiao et al., 2022). In the linear fast greedy block Kaczmarz method, an analogous averaged update uses $F(x)=0,\qquad x\in D\subset\mathbb{R}^n,$8 rather than $F(x)=0,\qquad x\in D\subset\mathbb{R}^n,$9, with the same computational rationale (Xiao et al., 2022).

The method therefore differs from pseudoinverse-based randomized block subsampling Kaczmarz–Motzkin schemes, which sample $F=(F_1,\dots,F_m)^T$0 rows, greedily retain $F=(F_1,\dots,F_m)^T$1 rows with largest residuals, and then compute

$F=(F_1,\dots,F_m)^T$2

before updating $F=(F_1,\dots,F_m)^T$3 (Zhang et al., 2022). RGFBK preserves the random-then-greedy logic but replaces the expensive block solve by a single Jacobian-transpose multiplication (Ding et al., 13 Aug 2025).

3. Pseudoinverse avoidance and the “fast” update

The defining computational innovation of nonlinear RGFBK is its avoidance of the classical block Kaczmarz step

$F=(F_1,\dots,F_m)^T$4

that is, the Moore–Penrose pseudoinverse of the Jacobian submatrix applied to the selected residual block (Ding et al., 13 Aug 2025). Instead, the method solves a one-dimensional weighted residual minimization and obtains an update of the form

$F=(F_1,\dots,F_m)^T$5

which requires only one matrix–vector product $F=(F_1,\dots,F_m)^T$6 and no pseudoinverse (Ding et al., 13 Aug 2025).

The computational significance is explicit in the source formulation: this reduces per-iteration cost, especially when $F=(F_1,\dots,F_m)^T$7 is large, and is better suited for high-dimensional problems (Ding et al., 13 Aug 2025). The same contrast appears in linear fast greedy block Kaczmarz analysis, where pseudoinverse-based block updates can cost $F=(F_1,\dots,F_m)^T$8 or more if computed by SVD, whereas the fast averaged alternative replaces them by sparse matrix–vector operations (Xiao et al., 2022).

A closely related distinction separates RGFBK from block nonlinear Kaczmarz methods in which the selected block $F=(F_1,\dots,F_m)^T$9 is updated through

$D$0

as in the BSNK1 and BSNK2 families for nonlinear systems (Zhang et al., 2022). Those methods establish linear convergence in expectation under the local tangential cone condition, but they retain the pseudoinverse bottleneck (Zhang et al., 2022). RGFBK can therefore be understood as a fast block analogue within the nonlinear Kaczmarz landscape.

This suggests a useful taxonomy. One branch comprises exact block-projection methods, which project onto the solution set of the chosen block and typically invoke a pseudoinverse (Zhang et al., 2022, Zhang et al., 2022). A second branch comprises fast or averaged methods, which replace exact block projection by a directionally scaled aggregate correction and thereby reduce arithmetic and memory overhead (Xiao et al., 2022, Ding et al., 13 Aug 2025). The nonlinear RGFBK method belongs to the second branch.

4. Stochastic greedy conditioning and convergence theory

The convergence analysis of nonlinear RGFBK is formulated in terms of the local tangential cone condition and a lower bound on the norms of the Jacobian rows $D$1, which are assumed to be bounded below by some $D$2 (Ding et al., 13 Aug 2025). If $D$3 is a solution of $D$4, then for the update choice $D$5 and a relaxation parameter satisfying

$D$6

where $D$7 comes from the cone condition, the iterates satisfy

$D$8

(Ding et al., 13 Aug 2025).

The quantity controlling the rate is the stochastic greedy condition number. Let

$D$9

For a matrix $F'(x)\in\mathbb{R}^{m\times n}$0, define

$F'(x)\in\mathbb{R}^{m\times n}$1

In the nonlinear method one substitutes $F'(x)\in\mathbb{R}^{m\times n}$2 (Ding et al., 13 Aug 2025).

The rate factor is therefore

$F'(x)\in\mathbb{R}^{m\times n}$3

The optimal choice $F'(x)\in\mathbb{R}^{m\times n}$4 yields

$F'(x)\in\mathbb{R}^{m\times n}$5

(Ding et al., 13 Aug 2025).

The proof strategy begins from a descent lemma showing

$F'(x)\in\mathbb{R}^{m\times n}$6

for $F'(x)\in\mathbb{R}^{m\times n}$7; conditional expectation over the random greedy choice $F'(x)\in\mathbb{R}^{m\times n}$8, the cone condition, and singular-value bounds then introduce $F'(x)\in\mathbb{R}^{m\times n}$9 (Ding et al., 13 Aug 2025).

This mode of analysis parallels earlier randomized block Kaczmarz theory in the linear case, where convergence rates depend on geometric properties of sampled submatrices and on block conditioning (Necoara, 2019). It also echoes the expectation-based convergence results of nonlinear greedy randomized sampling Kaczmarz methods, which obtain linear convergence factors involving cone-condition constants and Jacobian-dependent quantities for both single-sample and block methods (Zhang et al., 2022). The distinctive feature of RGFBK is that conditioning is encoded through a stochastic greedy condition number tailored to random-then-greedy block formation (Ding et al., 13 Aug 2025).

5. Parameterization, practical tuning, and large-scale use

The core algorithmic parameters are the control-sample size $I\subseteq[m]:=\{1,\dots,m\}$0, the greedy-block size $I\subseteq[m]:=\{1,\dots,m\}$1, and the relaxation parameter $I\subseteq[m]:=\{1,\dots,m\}$2 (Ding et al., 13 Aug 2025). Their practical roles are distinct. The sample size $I\subseteq[m]:=\{1,\dots,m\}$3 governs how much random exploration occurs at each iteration; the block size $I\subseteq[m]:=\{1,\dots,m\}$4 controls how many high-residual equations are aggregated into the update; and $I\subseteq[m]:=\{1,\dots,m\}$5 scales the correction (Ding et al., 13 Aug 2025).

For the nonlinear method, the stated convergence requirement is $I\subseteq[m]:=\{1,\dots,m\}$6 if $I\subseteq[m]:=\{1,\dots,m\}$7 is known, and the practical guidance is that $I\subseteq[m]:=\{1,\dots,m\}$8 often leads to good performance with $I\subseteq[m]:=\{1,\dots,m\}$9 (Ding et al., 13 Aug 2025). If an estimate of $$F_I(x)=(F_i(x))_{i\in I},\qquad F'_I(x)\in\mathbb{R}^{|I|\times n}$$0 is unavailable, a safe default is $$F_I(x)=(F_i(x))_{i\in I},\qquad F'_I(x)\in\mathbb{R}^{|I|\times n}$$1, and one may perform a short line-search on $$F_I(x)=(F_i(x))_{i\in I},\qquad F'_I(x)\in\mathbb{R}^{|I|\times n}$$2 (Ding et al., 13 Aug 2025). The source also recommends selecting $$F_I(x)=(F_i(x))_{i\in I},\qquad F'_I(x)\in\mathbb{R}^{|I|\times n}$$3 moderately large, for example $$F_I(x)=(F_i(x))_{i\in I},\qquad F'_I(x)\in\mathbb{R}^{|I|\times n}$$4 to $$F_I(x)=(F_i(x))_{i\in I},\qquad F'_I(x)\in\mathbb{R}^{|I|\times n}$$5 of $$F_I(x)=(F_i(x))_{i\in I},\qquad F'_I(x)\in\mathbb{R}^{|I|\times n}$$6, and $$F_I(x)=(F_i(x))_{i\in I},\qquad F'_I(x)\in\mathbb{R}^{|I|\times n}$$7 to $$F_I(x)=(F_i(x))_{i\in I},\qquad F'_I(x)\in\mathbb{R}^{|I|\times n}$$8 of $$F_I(x)=(F_i(x))_{i\in I},\qquad F'_I(x)\in\mathbb{R}^{|I|\times n}$$9, in order to balance block update power against cost (Ding et al., 13 Aug 2025).

Memory-aware implementation is emphasized. Only the Jacobian rows for indices in $F'_I(x_k)^T\omega_k$0 need to be precomputed, stored, or computed efficiently, which helps keep memory under control (Ding et al., 13 Aug 2025). Monitoring $F'_I(x_k)^T\omega_k$1 and adapting $F'_I(x_k)^T\omega_k$2 downward if stagnation or oscillation appears is also recommended (Ding et al., 13 Aug 2025).

These guidelines are consistent with the broader Kaczmarz literature. Randomized block subsampling methods report a U-shaped total-time curve when varying sample and greedy block sizes: too small a block yields many iterations, while too large a block raises per-iteration cost (Zhang et al., 2022). Fast greedy block Kaczmarz for linear systems similarly notes that smaller selection thresholds produce larger blocks and alter the balance between progress and cost (Xiao et al., 2022). This suggests that the empirical tuning rules in RGFBK reflect a general block-Kaczmarz trade-off rather than an isolated peculiarity of the nonlinear method.

6. Numerical behavior and relation to adjacent methods

The nonlinear RGFBK paper reports experiments on two large-scale test problems: the Chandrasekhar $F'_I(x_k)^T\omega_k$3-equation, with dense Jacobians and discretized dimension $F'_I(x_k)^T\omega_k$4 up to $F'_I(x_k)^T\omega_k$5, and the Broyden tridiagonal function, with sparse Jacobian and $F'_I(x_k)^T\omega_k$6 up to $F'_I(x_k)^T\omega_k$7 (Ding et al., 13 Aug 2025). The stopping criterion is

$F'_I(x_k)^T\omega_k$8

or $F'_I(x_k)^T\omega_k$9 iterations, and the reported parameter choice for RGFBK is $f(x)=0$0, $f(x)=0$1, $f(x)=0$2 (Ding et al., 13 Aug 2025).

The comparison set includes MR-BSNK, MD-BSNK, RB-CNK, and RBWNK (Ding et al., 13 Aug 2025). According to the reported summary, RGFBK required roughly half the iterations and CPU time of the best competing block methods on the $f(x)=0$3-equation for $f(x)=0$4 up to $f(x)=0$5. On the Broyden problem, competing methods either diverged or slowed dramatically as $f(x)=0$6 grew, while RGFBK remained robust and grew slowly in CPU. A parameter study further identified an optimal trade-off around $f(x)=0$7, $f(x)=0$8 (Ding et al., 13 Aug 2025).

The source interprets these outcomes through the stochastic greedy condition number and the relaxation parameter: when the Jacobian matrix exhibits a favorable stochastic greedy condition number and an appropriate relaxation parameter is selected, convergence is significantly accelerated (Ding et al., 13 Aug 2025). A plausible implication is that the method derives practical gains not only from cheaper iterations but also from more effective submatrix selection.

The performance claims fit a broader empirical pattern across related methods. In the linear setting, fast greedy block Kaczmarz often uses fewer CPU seconds and fewer iterations than earlier greedy or deterministic block Kaczmarz schemes (Xiao et al., 2022). Randomized block subsampling Kaczmarz–Motzkin methods outperform greedy block methods without random subsampling when $f(x)=0$9 is tuned well (Zhang et al., 2022). Nonlinear greedy randomized sampling methods report that block methods outperform single-sample variants and that greedy selection rules give faster convergence rates than random ones in applications including the Brown almost linear function and generalized linear model (Zhang et al., 2022). The nonlinear RGFBK results therefore extend an established empirical trend into the pseudoinverse-free nonlinear block regime (Ding et al., 13 Aug 2025).

7. Historical lineage, distinctions, and common misconceptions

The phrase “Random Greedy Fast Block Kaczmarz” can refer to more than one nearby construction, and distinguishing these is essential. The 2025 nonlinear method is specifically a solver for large-scale nonlinear systems $F:\mathbb{R}^n\to\mathbb{R}^m$00 that uses random sampling, greedy block selection, and a pseudoinverse-free update based on $F:\mathbb{R}^n\to\mathbb{R}^m$01 (Ding et al., 13 Aug 2025). Earlier fast greedy block Kaczmarz work addressed consistent linear systems $F:\mathbb{R}^n\to\mathbb{R}^m$02, with a fully greedy thresholding rule and an averaged update $F:\mathbb{R}^n\to\mathbb{R}^m$03, but without the initial random subsampling stage that defines the nonlinear RGFBK sampling mechanism (Xiao et al., 2022).

A second source of confusion is the relation to randomized block subsampling Kaczmarz–Motzkin methods. Those methods also sample a subset of rows, greedily retain the largest residuals, and update on a block; however, the update is a classical block projection using $F:\mathbb{R}^n\to\mathbb{R}^m$04 (Zhang et al., 2022). The 2025 nonlinear RGFBK method explicitly avoids that pseudoinverse (Ding et al., 13 Aug 2025). Referring to both methods as “random greedy block Kaczmarz” captures a family resemblance, but obscures the computational distinction between exact block projection and fast averaged correction.

A third distinction concerns the notion of “greedy.” In some Kaczmarz papers, greediness means choosing the maximum residual, as in Motzkin-type rules (Zhang et al., 2020, Zhang et al., 2022). In others, it refers to selecting all rows above a threshold derived from the maximum scaled residual (Xiao et al., 2022). In RGFBK, the greedy rule is top-$F:\mathbb{R}^n\to\mathbb{R}^m$05 selection among a random control set $F:\mathbb{R}^n\to\mathbb{R}^m$06 according to largest $F:\mathbb{R}^n\to\mathbb{R}^m$07 (Ding et al., 13 Aug 2025). These are related but not identical selection logics.

Finally, the method should not be conflated with generic randomized block Kaczmarz frameworks that rely primarily on well-conditioned row pavings and extrapolated stepsizes. Faster randomized block Kaczmarz theory shows that well-conditioned blocks and extrapolated stepsizes can yield linear convergence in expectation with rates governed by stochastic block geometry (Necoara, 2019). RGFBK shares the dependence on submatrix conditioning, but its central object is the stochastic greedy condition number induced by random-then-greedy selection, and its analysis is tied to the nonlinear cone-condition setting (Ding et al., 13 Aug 2025).

Taken together, these distinctions situate Random Greedy Fast Block Kaczmarz as a specific member of a broader Kaczmarz family: a pseudoinverse-free, random-then-greedy block method whose nonlinear 2025 formulation combines computational economy with expectation-linear convergence guarantees and strong large-scale empirical behavior (Ding et al., 13 Aug 2025).