Oblivious Subspace Injection Is Not Enough for Relative Error

Abstract: Oblivious subspace injection (OSI) was introduced by Camaño, Epperly, Meyer, and Tropp in 2025 as a much weaker sketching property than oblivious subspace embedding (OSE) that still yields constant-factor guarantees for randomized low-rank approximation and sketch-and-solve least-squares regression. At the Simons Institute in Berkeley during a workshop in October 2025, it was asked whether OSIs also imply relative error bounds rather than just constant-factor guarantees. We show that, from a theoretical standpoint, OSI alone does not yield OSE-style relative-error guarantees whose failure probability is controlled solely by the OSI failure parameter, even though OSI sketches often perform extremely well in practice. We provide counterexamples showing this for sketch-and-solve least squares and for randomized SVD in the Frobenius norm. The missing ingredient from a sketch satisfying only OSI is upper control on the optimal residual or tail component, and when one ensures the sketch has this additional property, a near-relative-error bound is recovered. We also show that there is a natural $\ell_p$ analogue of OSI giving constant-factor sketch-and-solve bounds.

• The paper reveals that OSI cannot provide OSE-style guarantees for relative error in sketching algorithms.
• It presents explicit counterexamples in least squares and randomized SVD to illustrate OSI's shortcomings in controlling the residual.
• The work shows that strengthening OSI with augmented subspace injectivity restores near-relative error bounds for practical applications.

Oblivious Subspace Injection Is Not Enough for Relative Error: A Technical Analysis

Introduction and Context

The study critically examines the theoretical limitations of the Oblivious Subspace Injection (OSI) property as a sketching tool in randomized numerical linear algebra. While OSI was recently introduced as a weaker alternative to the classical Oblivious Subspace Embedding (OSE) property—enabling constant-factor guarantees for randomized low-rank approximation and sketch-and-solve least squares regression—this work demonstrates that OSI does not suffice to yield OSE-style relative error bounds when one requires that the failure probability be controlled exclusively by the OSI parameter. The analysis is grounded in explicit counterexamples for both sketch-and-solve least squares and randomized SVD, and clarifies the precise technical distinction between OSI and OSE.

OSI vs. OSE: Formal Relationships and Parameter Gaps

Oblivious Subspace Embedding provides two-sided control over the geometry of all subspaces up to a certain dimension, guaranteeing that any vector in those subspaces exhibits a near-isometric transformation under the sketch. In contrast, OSI imposes only isotropy (preservation of norm in expectation) and injectivity (a one-sided lower bound) for each subspace.

The paper formally establishes that any OSI sketch can be upgraded to an OSE, but only with greatly weakened upper-distortion parameters. Specifically, for an $(s,\alpha,\rho)$-OSI and any $0<\tau<1-\rho$, the resulting OSE has upper-distortion parameter $\beta = \alpha + s(1-\alpha+\alpha\rho)/\tau$ and failure probability $\rho + \tau$. This deterioration is not an artifact of the proof, but is shown to be tight: it is possible to construct pathological OSI sketches that concentrate distortion into a single direction, saturating this bound. Consequently, no relative-error guarantee with controlled failure probability can be transferred from OSI to OSE without significant loss in parameter quality.

Counterexamples in Least Squares and Randomized SVD

A central contribution of the work is a series of explicit constructions demonstrating the insufficiency of OSI for relative error in both sketch-and-solve least squares and randomized SVD.

Least Squares: For any desired OSI failure parameter $\rho$, there exist sketches and problem instances where sketch-and-solve can incur a constant-factor loss with probability $\rho$, regardless of how close the injectivity parameter $\alpha$ is to 1. Even globally injective sketches (zero OSI failure probability) can exhibit a constant-factor gap with probability $\Omega(\epsilon)$ for arbitrarily small $\epsilon$. The failure arises because, although OSI controls the lower singular values on the column space of $A$, it provides no upper control on the component orthogonal to $0<\tau<1-\rho$0—the residual direction crucial for the least squares problem (2604.10215).

Randomized SVD: Analogously, in the context of randomized SVD in the Frobenius norm, the OSI property ensures injectivity on the leading singular subspace but does not control the interaction with the tail singular vectors. The paper demonstrates that this can lead—on minimal and natural problem instances—to an error factor of $0<\tau<1-\rho$1 in the approximation, confirming that OSI is inadequate for relative error bounds in low-rank approximation as well.

Recovery of Relative Error With Stronger Assumptions

A constructive analysis shows that if one strengthens the OSI property to ensure injectivity not just on the range of $0<\tau<1-\rho$2, but on the augmented subspace $0<\tau<1-\rho$3 for regression or $0<\tau<1-\rho$4 for each trailing singular vector $0<\tau<1-\rho$5 in SVD, near-relative error bounds can be recovered. The success probability in this regime depends on both the OSI injectivity parameter and the probability of simultaneous injectivity across several subspaces, typically leading to a failure probability of $0<\tau<1-\rho$6 for SVD with $0<\tau<1-\rho$7 trailing singular vectors.

This delineates the core theoretical implication: the missing ingredient in OSI (compared to OSE) is a lack of upper control on the residual or the tail. Isotropy provides only average-case upper bounding, and Markov-type arguments induce losses in success probability or approximation quality.

OSI Analogues for $0<\tau<1-\rho$8 Regression

The study extends the framework to $0<\tau<1-\rho$9 regression, introducing a $\beta = \alpha + s(1-\alpha+\alpha\rho)/\tau$0-isotropic OSI property (expectation preservation of the $\beta = \alpha + s(1-\alpha+\alpha\rho)/\tau$1-th moment under the sketch) and demonstrating that, in this setting, OSI continues to suffice for constant-factor approximation guarantees. A deterministic argument linking lower injectivity on $\beta = \alpha + s(1-\alpha+\alpha\rho)/\tau$2 and upper control on the residual immediately yields probabilistic guarantees through Markov's inequality, as long as the residual can be bounded in expectation.

Numerical and Practical Observations

Empirical evaluations indicate that, for practical problems, many OSI sketches (especially popular structured or fast sketches) perform comparably to OSE sketches with respect to relative error. The theoretical distinction may thus be most significant in worst-case analysis, for highly adversarial input configurations, or for applications requiring strong probabilistic guarantees with minimal loss factors.

Implications and Directions for Future Research

The findings clarify the rigorous boundaries between OSI and OSE-based sketching in randomized linear algebra. While OSI enables algorithmic efficiency and easily accommodates structured or fast sketches—broadening the applicability of sketching-based solvers—OSE-style relative error guarantees remain unattainable purely under the OSI definition. Future research may focus on identifying minimal practical conditions or hybrid sketch properties that interpolate between OSI and OSE, balancing provable relative error with efficient structured sketches. There is also interest in developing sharper lower bounds, or in understanding the empirical risk of worst-case configurations for both OSI sketches in large-scale and high-dimensional regimes.

Conclusion

The paper establishes that OSI is too weak, in general, to ensure OSE-style relative error guarantees for randomized least squares and low-rank approximation. The technical gap is precisely characterized, with explicit counterexamples and parameter-optimal negative results, but it is shown that modestly stronger injectivity assumptions on relevant augmented subspaces are sufficient to restore near-relative error guarantees. These results inform both theory and the high-performance implementation of randomized algorithms in numerical linear algebra, where a careful choice of sketching property is critical for balancing efficiency, structure, and approximation quality.

---

Reference: "Oblivious Subspace Injection Is Not Enough for Relative Error" (2604.10215)