Create and explore imaginative 3D worlds with our new product Dreamfield (opens in a new tab)

Theoretical analysis of comparable test performance among block preconditioners

Establish a rigorous theoretical analysis explaining why the standard Block Jacobi preconditioner, the hybrid block preconditioner, and the BLOKS sketched block Jacobi preconditioner exhibit almost indistinguishable test-time performance, and determine whether regularization effects caused by block approximation account for this phenomenon.

Background

The paper compares three preconditioning strategies for Nyström-sketched Kernel Ridge Regression: the standard Block Jacobi preconditioner, a hybrid preconditioner based on a block approximation of the Falkon preconditioner, and the BLOKS sketched block Jacobi preconditioner. The standard Block Jacobi method retains more information in each block but is computationally expensive, whereas BLOKS sketches each block to achieve memory-efficient computation.

Empirical experiments show that the more informative preconditioners can yield better training-time convergence, but their test-time performances are nearly indistinguishable. The authors suggest that approximating the blocks may introduce additional regularization and that the information lost relative to standard Block Jacobi may be compensated for by an appropriate choice of the regularization parameter. A rigorous explanation of this behavior is not provided and is explicitly left unresolved.

References

We believe this phenomenon might be partly caused by additional regularization introduced by the approximation of the blocks, and we leave a rigorous theoretical analysis for future work.

— BLOKS: Scaling Kernel Solvers with Sketched Block Preconditioning  (2610.13382 - Vitale et al., 9 Oct 2026) in Appendix, Section “Block preconditioning strategies and comparisons” (Section \ref{apx:block_prec})