Deterministic Geometric Balancing Map

• The paper introduces a deterministic post-processing method that collapses all nonzero singular values to their geometric mean, ensuring uniform conditioning.
• It employs a thin SVD of the sketched matrix combined with a diagonal balancing operator, followed by a reconstruction that scales the column space exactly.
• Empirical results show improved iterative solver performance and reduced iteration counts, making the method valuable for high-dimensional regression and numerical linear algebra.

A deterministic geometric balancing map is a post-processing operator designed to regularize the singular value spectrum of a sketched matrix, ensuring all nonzero singular values collapse to a common geometric mean. This concept emerges in the context of oblivious sketching and subspace embeddings, where one seeks dimensionality reduction with provable norm-preservation properties for all vectors in a subspace—especially in high-dimensional numerical linear algebra, regression, and fast iterative solvers. The deterministic geometric balancing map, when composed with a sparse oblivious sketching matrix, enables control of the condition number of the resulting sketched matrix under structural assumptions such as bounded condition number and incoherence of the original matrix (Mango et al., 16 Nov 2025).

1. Background: Sparse Oblivious Subspace Embeddings and Conditioning

Oblivious subspace embeddings (OSEs) provide data-independent linear projections $\Pi\in\mathbb{R}^{m\times n}$ that, for all vectors $x$ in a target $d$-dimensional subspace $T$, satisfy: $$\Pr_\Pi\left[\forall x\in T,\, (1-\epsilon)\|x\|_2 \leq \|\Pi x\|_2 \leq (1+\epsilon)\|x\|_2\right] \geq 1-\delta$$ OSEs such as CountSketch and OSNAP are chosen to be extremely sparse for input-sparsity multiplication time. Fundamental lower bounds show that, for sparsity $s=1$ (one nonzero per column), the number of rows $m$ must scale as $\Omega(d^2/(\epsilon^2\delta))$ (Li et al., 2021). These embeddings guarantee all singular values of the mapped subspace are within a $1\pm\epsilon$ envelope.

Yet these bounds do not guarantee “collapse” or uniformity of the singular value spectrum—a potential issue for downstream numerical stability, as iterative methods and regression often benefit from matrices with controlled condition number.

2. Construction of the Deterministic Geometric Balancing Map

Given a sketched matrix $B = \Phi A$, with $\Phi$ a sparse OSE, $A\in\mathbb{R}^{m\times n}$ has rank $r$, and $\Phi\in\mathbb{R}^{s\times m}$, the deterministic geometric balancing map is as follows (Mango et al., 16 Nov 2025):

1. Compute thin SVD of $B$:

$$B = \widetilde U\,\widetilde \Sigma\,\widetilde V^\top,$$

where $$\widetilde \Sigma = \mathrm{diag}(\widetilde \sigma_1,\ldots,\widetilde \sigma_r)$$.

2. Geometric mean of the nonzero singular values:

$$\widetilde g = \left(\prod_{i=1}^r \widetilde \sigma_i\right)^{1/r}.$$

3. Construct diagonal “balancing” matrix:

$$\Lambda = \mathrm{diag}\left(\frac{\widetilde g}{\widetilde \sigma_1},\ldots,\frac{\widetilde g}{\widetilde \sigma_r}\right).$$

4. Form the symmetric balancing operator:

$$W = \widetilde U\,\Lambda\,\widetilde U^\top + (I_s - \widetilde U\,\widetilde U^\top).$$

The action of $W$ scales the directions aligned with the column space of $B$ by the corresponding weights, and acts as identity outside it.

5. Final “balanced” sketch:

$S = W\,\Phi.$

Applying $S$ to $A$ produces:

$$SA = W\,B = \widetilde U\, (\Lambda\,\widetilde \Sigma)\, \widetilde V^\top,$$

but by construction, $\Lambda\,\widetilde \Sigma = \widetilde g\,I_r$, so all nonzero singular values of $SA$ are exactly $\widetilde g$.

This exact spectral uniformity is unattainable for truly oblivious sketches (where $W$ would depend on $A$), but this two-stage procedure is deterministic once $\Phi$ and $A$ are fixed.

3. Theoretical Guarantee and Limitations

Suppose $A$ has rank $r$ with $\kappa(A):= \sigma_1(A)/\sigma_r(A)$ bounded and the column-space coherence is $O(1)$. For $\epsilon\in(0,1/4)$ and $\delta\in(0,1/10)$, there exists an absolute constant $C$ for which, if $s\geq C\,\epsilon^{-2}\,r\,\log(r/\delta)$ and $\Phi$ is a sparse OSE (e.g., CountSketch or OSNAP), then with probability $\geq 1-\delta$:

• Every nonzero singular value of $SA$ satisfies

$$\frac{1-\epsilon}{\kappa_0} \sigma_i(A) \leq \sigma_i(SA) \leq (1+\epsilon) \kappa_0 \sigma_i(A)$$

for all $i=1,\ldots,r$, and $\kappa(SA)=1$ exactly.

Stability with respect to perturbations $A+E$ with $\|E\|_2\leq \eta\,\sigma_r(A)$ is also controlled, with singular values perturbed by $O(\eta\,\sigma_r(A))$.

Oblivious lower bounds remain valid: for a fully oblivious $S$, achieving uniform relative error on all singular values for all rank-$r$ input matrices requires $s\geq c_0\,(r+\log(1/\delta))/\epsilon^2$. Thus, the geometric balancing map cannot circumvent classical Johnson–Lindenstrauss-type lower bounds in the worst case (Mango et al., 16 Nov 2025).

4. Empirical Performance and Practical Implications

Empirical studies show that geometric balancing achieves exact condition number 1 for “good” (low coherence, mild spectrum decay) matrices, reducing both extremal singular value distortion and improving iterative solver convergence (e.g., Krylov, gradient descent) (Mango et al., 16 Nov 2025). On high-coherence or nearly rank-deficient inputs, extreme singular value inflation may occur, but significant condition-number reduction is still observed compared to classical sketches.

In large-scale least squares or PDE numerical solvers, geometric balancing leads to

• Reduced iteration counts (e.g., 20–38% fewer steps than CountSketch).
• Near-perfect residuals at target tolerances.

This suggests deterministic balancing can be valuable for applications where stable conditioning is critical and input structure is favorable.

5. Comparison to Classical and Sparse Oblivious Sketches

Method	Sketch Dim. $s$	Uniform Extremal Control	Requires Data-Dependent Step
CountSketch	$\Theta(r^2)$	No	No
OSNAP	$\Theta(r\,\log r)$	No	No
Balancing Map (W)	$O(r\,\log r)$ (plus SVD step)	Yes (under $\kappa_0$)	Yes (deterministic, post-sketch)

While CountSketch and OSNAP offer input-sparsity-time embedding but only $(1\pm\epsilon)$ control per singular value, the balancing map provides uniform scaling, strictly within the image of $A$ under $\Phi$, but requires the SVD of $\Phi A$.

6. Scope and Limitations

The deterministic geometric balancing map achieves its strongest guarantees when the input matrix has bounded condition number and low coherence. In adversarial settings (e.g., nearly rank-deficient or high-coherence inputs), the balancing may amplify small singular values outside the desired range, but condition number is still reduced. No oblivious sketch can guarantee exact control of all singular values across all possible rank-$r$ matrices below Johnson–Lindenstrauss lower bounds.

A plausible implication is that deterministic geometric balancing is best-suited as a post-processing step in structured or favorably-conditioned domains, and should not be used as a universal replacement for fully oblivious subspace embeddings.

7. Applications and Future Directions

Deterministic geometric balancing maps are applicable wherever downstream algorithms are sensitive to the condition number of the sketched matrix:

• Accelerating convergence of iterative linear solvers.
• Improving numerical stability in low-rank approximation and regression problems.
• Regularizing data-dependent sketches in distributed or streaming algorithms, provided SVD computation on sketched outputs is feasible.

Open questions pertain to reducing the cost of the balancing step and to developing variants that can handle higher coherence or nearly rank-deficient cases with provable uniform control (Mango et al., 16 Nov 2025). The interplay between sketch structure, input matrix geometry, and deterministic post-processing defines a current frontier in fast, stable randomized numerical linear algebra.