Quantile-Based Randomized Kaczmarz (QRK)

• Quantile-based Randomized Kaczmarz (QRK) is a robust iterative algorithm that filters row updates using residual quantile thresholds to mitigate sparse, gross corruptions in overdetermined systems.
• It computes empirical q-quantiles of absolute residuals to selectively update by avoiding corrupted hyperplanes, thereby preserving linear convergence.
• QRK’s performance is tunable via quantile parameters and subsampling strategies, making it adaptable to various noise and corruption levels in practical applications.

Quantile-based Randomized Kaczmarz (QRK), also called QuantileRK or qRK, is a robust Kaczmarz-family row-action method for large, overdetermined linear systems in which row updates are filtered by residual quantiles. Its central purpose is to solve systems whose measurement vector contains sparse, arbitrarily large corruptions, optionally together with widespread small-scale noise, without being driven toward corrupted hyperplanes. In the standard formulation, QRK computes a threshold from the empirical distribution of absolute residuals and accepts only rows whose residual magnitude is below that threshold; under suitable conditioning and corruption-fraction assumptions, this preserves linear convergence while suppressing the effect of outliers (Jarman et al., 2021).

1. Problem model and conceptual basis

QRK is typically posed for a linear system with an underlying consistent component

$A x = \tilde b,$

where $A \in \mathbb{R}^{m\times n}$, $m \ge n$, and the target solution is $x_* \in \mathbb{R}^n$. The observed right-hand side is contaminated: $b = \tilde b + b^C + r,$ where $b^C$ is a sparse corruption vector and $r$ is a noise vector. The corruption fraction is

$\beta = \frac{|\{i : b^C_i \neq 0\}|}{m}.$

The standard analysis assumes row normalization, $\|a_i\|_2 = 1$, so that the Kaczmarz projection step takes its simplest form (Jarman et al., 2021).

The motivation is the failure mode of classical randomized Kaczmarz under sparse gross corruptions. Standard RK samples rows without regard to whether the corresponding equation is reliable; a projection onto a corrupted hyperplane can move the iterate far from $x_*$. By contrast, QRK uses order statistics of current residuals to exclude the largest residual rows, which are likely to be corrupted, and thereby attempts to recover the solution of the clean system rather than the least-squares solution of the contaminated one. This filtering viewpoint distinguishes QRK from methods that deliberately target large residuals for acceleration on clean systems (Steinerberger, 2021).

Two related conditioning notions recur in the theory. One is the restricted smallest singular value over sufficiently large row subsets, often written as $A \in \mathbb{R}^{m\times n}$0 or an analogous subset-conditioned quantity. The other is $A \in \mathbb{R}^{m\times n}$1. Their ratio governs whether the contraction induced by accepted clean rows dominates the expansion that can arise from occasionally accepting corrupted rows. This suggests that QRK is fundamentally a robust-selection method whose performance depends not only on the corruption fraction $A \in \mathbb{R}^{m\times n}$2 but also on how well conditioned the surviving clean submatrices remain.

2. Core algorithmic mechanism

At iterate $A \in \mathbb{R}^{m\times n}$3, QRK forms residuals

$A \in \mathbb{R}^{m\times n}$4

and computes quantiles on the absolute residuals $A \in \mathbb{R}^{m\times n}$5. In the basic notation,

$A \in \mathbb{R}^{m\times n}$6

and, for a subset $A \in \mathbb{R}^{m\times n}$7,

$A \in \mathbb{R}^{m\times n}$8

The accepted rows are those with residual magnitude no larger than the chosen quantile threshold (Jarman et al., 2021).

A standard accept-if formulation proceeds as follows. At iteration $A \in \mathbb{R}^{m\times n}$9, sample a set of $m \ge n$0 indices $m \ge n$1 uniformly from $m \ge n$2, compute the sub-residual quantile $m \ge n$3, sample a candidate row $m \ge n$4 uniformly from $m \ge n$5, and accept the row if

$m \ge n$6

If accepted, the normalized-row update is

$m \ge n$7

otherwise $m \ge n$8. In this form, both the $m \ge n$9-row subsample and the candidate row are sampled uniformly, and the acceptance probability is approximately $x_* \in \mathbb{R}^n$0 by construction (Jarman et al., 2021).

Later analyses distinguish two closely related sampling variants. In QuantileRK1, one samples $x_* \in \mathbb{R}^n$1 uniformly from $x_* \in \mathbb{R}^n$2 and updates only if $x_* \in \mathbb{R}^n$3 lies in the admissible set

$x_* \in \mathbb{R}^n$4

so the per-iteration acceptance probability is $x_* \in \mathbb{R}^n$5. In QuantileRK2, one samples directly from $x_* \in \mathbb{R}^n$6, so $x_* \in \mathbb{R}^n$7. In both cases, once an admissible row is chosen, the update is the usual Kaczmarz projection (Coria et al., 2024).

The algorithmic distinction from greedy residual methods is structural. QRK keeps rows with small residuals, not large ones. Reverse-quantile variants invert this rule and can accelerate convergence on clean consistent systems, but they do not inherit QRK’s corruption robustness. This difference is not cosmetic: in the sparse-corruption setting, selecting large residual rows is exactly the behavior QRK is designed to avoid (Battaglia et al., 2024).

3. Convergence theory

A deterministic QRK guarantee can be stated directly in terms of matrix-dependent subset conditioning. For normalized rows, sparse adversarial corruption with fraction $x_* \in \mathbb{R}^n$8, and a quantile parameter $x_* \in \mathbb{R}^n$9, linear convergence in expectation holds if

$b = \tilde b + b^C + r,$0

Under this inequality there exists $b = \tilde b + b^C + r,$1 such that

$b = \tilde b + b^C + r,$2

This formulation makes explicit that QRK’s robustness is controlled by the competition between clean-subset contraction and the worst-case damage caused by admissible corrupted rows (Steinerberger, 2021).

The 2021 noise analysis extends the corruption-only theory to the model $b = \tilde b + b^C + r,$3, where $b = \tilde b + b^C + r,$4 may affect every row and no distributional assumption is imposed in the theory beyond bounds through $b = \tilde b + b^C + r,$5. Under standardized random-matrix assumptions—independent isotropic subgaussian rows, an entrywise density bound, and a tall system $b = \tilde b + b^C + r,$6—the full-quantile iterate satisfies

$b = \tilde b + b^C + r,$7

The exponential factor matches the corruption-only case up to constants, while the additive term is a noise floor proportional to $b = \tilde b + b^C + r,$8. In the noiseless corrupted case, the additive horizon disappears (Jarman et al., 2021).

A further extension allows time-varying perturbations,

$b = \tilde b + b^C + r,$9

with $b^C$0 at each iteration. For QuantileRK1 and QuantileRK2 there are explicit constants $b^C$1 and $b^C$2 such that the rate is $b^C$3, with $b^C$4 for QuantileRK1 and $b^C$5 for QuantileRK2; the asymptotic horizon depends on the first and second absolute moments of the noise. The same analysis also yields a Markov-inequality-based lower bound on the probability that the largest residual entries reveal the current corrupted indices, provided the corruption magnitude dominates the noise scale in the prescribed sense (Coria et al., 2024).

These results support a characteristic interpretation of QRK. In pure sparse-corruption models, QRK is a linearly convergent exact-recovery method under suitable conditioning. In mixed corruption-plus-noise models, it is a linearly convergent robust estimator with an explicit convergence horizon. A common misconception is to treat quantile filtering as eliminating noise; the available theory instead shows that it eliminates the effect of sparse gross corruptions while retaining a noise-dependent floor whenever pervasive noise remains.

4. Parameter regimes, cost, and practical tuning

The main parameter is the quantile level $b^C$6. Smaller $b^C$7 imposes stricter acceptance and thus greater robustness, but also reduces the number of usable updates. Larger $b^C$8 increases the admissible set and usually accelerates convergence, but risks admitting corrupted rows if $b^C$9 approaches $r$0 too closely. In the 2021 analysis the requirement is

$r$1

whereas other formulations use the simpler admissibility window $r$2. Empirically, choosing $r$3 close to $r$4 was observed to work well and often accelerates convergence while maintaining robustness (Jarman et al., 2021).

When the quantile is estimated from a subsample of size $r$5, the dominant per-iteration work is computing $r$6 inner products and residual magnitudes, so the cost is $r$7, plus $r$8 expected quantile selection and $r$9 for the candidate-row test and possible update. The experiments in the 2021 noisy-corrupted paper use $\beta = \frac{|\{i : b^C_i \neq 0\}|}{m}.$0, and the implementation guidance explicitly recommends choosing $\beta = \frac{|\{i : b^C_i \neq 0\}|}{m}.$1 large enough to stabilize the quantile estimate but small enough for efficiency, for example in the hundreds (Jarman et al., 2021).

Subsampled quantile estimation has its own theory. One line of analysis studies sQRK with a sample fraction $\beta = \frac{|\{i : b^C_i \neq 0\}|}{m}.$2, requiring

$\beta = \frac{|\{i : b^C_i \neq 0\}|}{m}.$3

so that the accepted subsample still contains enough uncorrupted rows and the quantile remains informative. Under these constraints, sQRK converges linearly in expectation to the solution of the uncorrupted system when the derived rate constant is below $\beta = \frac{|\{i : b^C_i \neq 0\}|}{m}.$4 (Haddock et al., 2023). A later result sharpens the required subsample size dramatically: under standard scaling assumptions on $\beta = \frac{|\{i : b^C_i \neq 0\}|}{m}.$5, QRK with quantile subsample size

$\beta = \frac{|\{i : b^C_i \neq 0\}|}{m}.$6

linearly converges over the first $\beta = \frac{|\{i : b^C_i \neq 0\}|}{m}.$7 iterations with high probability, and this scaling is tight up to constants. A plausible implication is that robust quantile screening need not cost $\beta = \frac{|\{i : b^C_i \neq 0\}|}{m}.$8 per iteration in the large-$\beta = \frac{|\{i : b^C_i \neq 0\}|}{m}.$9 regime (Cai et al., 21 Jul 2025).

Implementation advice is consistent across formulations. Rows should be normalized whenever possible; if they are not, the update must be divided by $\|a_i\|_2 = 1$0. Quantiles may be computed by selection algorithms, partial sorting, or streaming sketches. Stopping criteria include a fixed iteration budget, stabilization of $\|a_i\|_2 = 1$1, or plateauing residual behavior near the predicted noise floor. In large-scale settings, the ability to access rows in a streamed or sampled fashion is one of QRK’s defining practical advantages (Jarman et al., 2021).

5. Variants and generalizations

A major extension replaces the single accepted-row update by an averaged block step. In QuantileABK and related block-averaged variants, one computes a quantile threshold, forms a trusted candidate set, and updates by averaging one-dimensional projections over a block $\|a_i\|_2 = 1$2. Under subgaussian-type assumptions, the resulting rate is constant in $\|a_i\|_2 = 1$3 and $\|a_i\|_2 = 1$4, yielding an $\|a_i\|_2 = 1$5-fold speed-up over single-row QRK in the canonical random-matrix regime. The same work also shows that the classical projection-based block Kaczmarz method is not robust to sparse adversarial corruptions; robustness requires averaging one-dimensional projections rather than enforcing a corrupted block intersection (Cheng et al., 2022).

QRK has also been integrated with sparse recovery via Bregman iterations and soft-thresholding. In Quantile-RaSK, sparsity is enforced through

$\|a_i\|_2 = 1$6

with updates performed in a dual variable and the primal iterate recovered by soft-thresholding. The block-averaged Quantile-RaSKA inherits the same corruption filtering and achieves a factor-$\|a_i\|_2 = 1$7 acceleration in the subgaussian setting. More recently, adaptive heavy-ball momentum selected by a minimal dual function principle has been combined with quantile-based sampling in a unified framework for exact, noisy, and corrupted sparse systems, with linear convergence in expectation of Bregman distance up to a finite horizon related to the contamination level (Zhang et al., 2022, Zhang et al., 31 Mar 2026).

Another extension incorporates external knowledge. QuantileSCRK imposes a hard subspace constraint $\|a_i\|_2 = 1$8 on a trusted subset $\|a_i\|_2 = 1$9 of corruption-free equations and applies quantile-based constrained Kaczmarz updates only on the remaining rows. In projector form, the update uses

$x_*$0

and replaces $x_*$1 by $x_*$2. On Gaussian-like data this creates an effective dimension reduction, improving the rate from the ambient dimension $x_*$3 to the reduced dimension $x_*$4 when $x_*$5 (Lok et al., 2023).

Several works study alternative quantile filters. Reverse Quantile-RK keeps rows above a quantile threshold and accelerates clean consistent problems by focusing on larger residuals, but it is explicitly not robust to corruption. Double-Quantile RK (dqRK) introduces a band-pass rule,

$x_*$6

which combines an upper quantile for robustness with a lower quantile for acceleration. The per-iteration cost is essentially the same as QRK because both quantiles can be obtained from the same sort or selection pass. A separate error-horizon analysis for qRK and dqRK shows smaller horizons than their non-quantile counterparts in mixed noise-plus-sparse-corruption regimes (Battaglia et al., 2024, Battaglia et al., 1 May 2025).

Other recent modifications include an online whitelist trust mechanism that maintains whitelist and blocklist sets, repeatedly removes rows with persistently large residuals, and adapts the quantile level according to the current effective corruption rate. Its theoretical rationale is that the QRK contraction factor improves monotonically as $x_*$7 decreases, so successful online elimination of unreliable rows should speed convergence (Shvaiko et al., 12 Feb 2026). Quantile gating has also been extended to tensor linear systems under the t-product. In QTRK, the quantile is computed over absolute tensor residual entries, entire row slices are excluded when any entry appears corrupted, and linear convergence in expectation is proved under the stated assumptions; the masked variant mQTRK performs partial column-slice updates but admits a pessimistic theory and explicit adversarial failure modes (Castillo et al., 23 Mar 2025).

6. Empirical behavior, comparisons, and limitations

In the noisy-corrupted Gaussian experiments of the 2021 QuantileRK paper, the system is $x_*$8 with standardized Gaussian rows, the corruption fraction is varied through $x_*$9, the corruption magnitudes are sampled from $A \in \mathbb{R}^{m\times n}$00, the noise entries are $A \in \mathbb{R}^{m\times n}$01, and the quantile estimate uses $A \in \mathbb{R}^{m\times n}$02 sampled rows. For fixed $A \in \mathbb{R}^{m\times n}$03 and $A \in \mathbb{R}^{m\times n}$04, larger corruptions relative to the noise are easier to detect and exclude, producing faster convergence; when corruptions are small, QRK still converges down to the predicted noise-driven horizon. The same study reports that the final error after 2000 iterations supports choosing $A \in \mathbb{R}^{m\times n}$05 very close to $A \in \mathbb{R}^{m\times n}$06, and that across 100 trials the observed plateau aligns closely with the theoretical horizon $A \in \mathbb{R}^{m\times n}$07 (Jarman et al., 2021).

Time-varying experiments show similar qualitative behavior. With $A \in \mathbb{R}^{m\times n}$08, normalized Gaussian rows, sparse supports that vary across iterations, and Gaussian noise levels $A \in \mathbb{R}^{m\times n}$09, QRK’s empirical convergence closely follows the theoretical upper bound for both static and time-varying perturbations. The reported corruption-detection experiments also show that the fraction of corrupted indices captured among the largest $A \in \mathbb{R}^{m\times n}$10 residual entries quickly approaches $A \in \mathbb{R}^{m\times n}$11, while the Markov-based lower bound remains conservative (Coria et al., 2024).

The whitelist mechanism supplies a complementary empirical narrative. On synthetic systems with $A \in \mathbb{R}^{m\times n}$12, $A \in \mathbb{R}^{m\times n}$13, and $A \in \mathbb{R}^{m\times n}$14, subsample WL-QRK converges faster than subsampled QRK across several corruption models, while standard RK does not converge meaningfully. On Wisconsin Breast Cancer and tomography systems, WL-QRK and QRK behave similarly during warm-up, after which the whitelist mechanism lowers the effective corruption rate and outpaces plain QRK. The same experiments report that subsampled WL-QRK performs similarly to full-sample WL-QRK at reduced per-iteration cost (Shvaiko et al., 12 Feb 2026).

The main limitations are consistent across the literature. Most sharp convergence results assume tall systems, full column rank, normalized rows, and either subgaussian-style random designs or explicit restricted-singular-value conditions. Guarantees deteriorate when $A \in \mathbb{R}^{m\times n}$15 becomes too large relative to $A \in \mathbb{R}^{m\times n}$16, because the admissible set can then be polluted by corrupted rows. Dense or heavy-tailed noise weakens the separation between corrupted and uncorrupted residuals, so QRK may converge only to a horizon rather than exactly. Extremely small quantile subsamples can be unstable, and projection-based block Kaczmarz remains non-robust in sparse adversarial settings even when combined with quantile screening (Jarman et al., 2021, Cheng et al., 2022, Haddock et al., 2023).

QRK is therefore best understood not as a universal cure for inconsistency, but as a family of robust row-selection mechanisms. Its defining contribution is to replace indiscriminate or large-residual row choice by a quantile-screened update rule that can preserve linear convergence in the presence of sparse gross corruption, while admitting a broad ecosystem of extensions—subsampled, block-averaged, sparse/Bregman, constrained, double-quantile, whitelist-based, and tensor-valued—that adapt the same principle to different geometric and computational regimes.