Entrywise Low-Rank Approximation and Matrix $p \rightarrow q$ Norms via Global Correlation Rounding

Abstract: Given a matrix $A$, the goal of the entrywise low-rank approximation problem is to find $\operatorname{argmin} |A-B|_p$ over all rank-$k$ matrices $B$, where $| \cdot |_p$ is the entrywise $\ell_p$ norm. When $p = 2$ this well-studied problem is solved by the singular value decomposition, but for $p \neq 2$ the problem becomes computationally challenging. For every even $p > 2$ and every fixed $k$, we give the first polynomial-time approximation scheme for this problem, improving on the $(3 + \varepsilon)$ approximation of Ban, Bhattiprolu, Bringmann, Kolev, Lee, and Woodruff, the bi-criteria approximation of Woodruff and Yasuda, and the additive approximation scheme of Anderson, Bakshi, and Hopkins. Prior algorithmic approaches based on sketching and column selection, which yielded a polynomial-time approximation scheme in the $p < 2$ setting, face concrete barriers when $p > 2$. Instead, we use the Sherali-Adams hierarchy of convex programs, and in so doing establish a blueprint for how to use convex hierarchies to design polynomial-time approximation schemes for continuous optimization problems. We use the same algorithmic strategy to give a new family of additive approximation algorithms for matrix $p \rightarrow q$ norms, which are intimately related to small-set expansion and quantum information. In particular, we give the first nontrivial additive approximation algorithms in the regime $p < 2 < q$.

• The paper introduces a polynomial-time approximation scheme for ℓp low-rank approximation (p>2) that achieves a (1+ε) multiplicative guarantee.
• It presents additive approximation algorithms for matrix p→q norms in hypercontractive regimes, tightly bounding errors with respect to row-masses.
• The work unifies advanced pinning lemmas with mirror descent to facilitate global correlation rounding under weak moment assumptions.

Entrywise Low-Rank Approximation and Matrix $p \rightarrow q$ Norms via Global Correlation Rounding

Problem Formulation and Prior Landscape

This work addresses two central problems in high-dimensional continuous optimization: entrywise low-rank matrix approximation in the $\ell_p$ norm for even $p > 2$ and the matrix $p \rightarrow q$ norm computation, especially in regimes where no efficient algorithms have previously been known.

• Entrywise Low-Rank Approximation ($\ell_p$-LRA): For a given $A \in \mathbb{R}^{n \times d}$, find a rank-$k$ matrix $B$ such that $\|A - B\|_p$ is minimized, measured entrywise. While SVD solves this when $p=2$, all $\ell_p$0 cases have resisted polynomial-time approximation schemes (PTAS) until now. Prior algorithms either offered constant-factor approximations, bicriteria bounds (outputting rank $\ell_p$1), or additive rather than multiplicative guarantees.
• Matrix $\ell_p$2 Norms: Given $\ell_p$3, compute $\ell_p$4. While $\ell_p$5 is efficient (largest singular value), variants with $\ell_p$6 (especially "hypercontractive" $\ell_p$7) are known to be NP-hard to even approximate. Prior guarantees were additive and often incurred polynomial factors in the error term.

Main Contributions

1. Polynomial-Time Approximation Scheme for $\ell_p$8 Entrywise $\ell_p$9-LRA

Theorem 1: For any even $p > 2$0 and fixed $p > 2$1, there is a PTAS (runtime $p > 2$2) delivering

$p > 2$3

when $p > 2$4 has integer entries of bounded bit complexity.

• This closes the gap for efficient $p > 2$5-approximate rank-$p > 2$6 approximation for $p > 2$7, where previous algorithms offered only multiplicative $p > 2$8-approximation or bicriteria schemes. This moves the tractability boundary for $p > 2$9 low-rank approximation to match the $p \rightarrow q$0 case, modulo parity of $p \rightarrow q$1.

2. Additive Approximation Algorithms for Matrix $p \rightarrow q$2 Norms

Theorem 2: For $p \rightarrow q$3 even, and $p \rightarrow q$4 with dual $p \rightarrow q$5 both even:

$p \rightarrow q$6

where $p \rightarrow q$7 is the $p \rightarrow q$8th row. The runtime and accuracy scales polynomially in $p \rightarrow q$9. This gives the first nontrivial additive approximation in the open regime $\ell_p$0, with error essentially matching the "row-mass" structure of $\ell_p$1.

• Previous additive approximations (e.g., [brandao2015replacing]) could incur additional polynomial factors or were restricted to $\ell_p$2.

3. A Unified Theory of Pinning Lemmas via Mirror Descent

The technical innovation is the use of Sherali-Adams (SA) or Sum-of-Squares relaxations and a general theory of "global correlation rounding" tied to advanced probabilistic decomposition results ("pinning lemmas"). The connection to mirror descent enables sharper control of heavy-tailed variables and non-Euclidean geometry, yielding stronger pinning lemmas that work under weaker moment assumptions.

• Previously, pinning lemmas (reducing correlations via conditioning) either relied on finite variance (variance-based) or bounded-support/entropy (discrete), which failed for heavy-tailed continuous distributions.
• The new heavy-tailed pinning lemma shows that after conditioning on $\ell_p$3 variables, pairwise covariances can be bounded in terms of pairwise low moments (rather than variances), even when variances do not exist.
• The general approach handles both additive and multiplicative error objectives, and is powerful enough to lift PTAS proofs from $\ell_p$4 to general $\ell_p$5 (rank).

Methodology

The algorithms are rooted in solving degree-$\ell_p$6 SA or SoS relaxations (which can be viewed as working with "pseudo-distributions" over variables) and then employing a global correlation rounding procedure. The key insight is to use pinning lemmas—measure decomposition theorems stemming from statistical physics and combinatorics—to control the error introduced when rounding (i.e., producing actual rank-$\ell_p$7 matrices or vectors from the relaxation).

• For LRA: The proof uses a series of new matrix inequalities, bootstrapping from a constant-factor approximate solution (using prior SDPTAS), and then arguing that within a small $\ell_p$8-ball about that solution, an optimal solution must exist—despite potential heavy tails in the residuals.
• For $\ell_p$9 norms: The rounding analysis controls the loss in objective from "product versus joint" expectations, using a new heavy-tailed pinning lemma to argue conditioning suffices to strongly decorrelate the pseudo-random variables.

The technical core (mirror descent-based potential reduction) generalizes previous approaches by operating in a space defined by Bregman divergences for functions such as $A \in \mathbb{R}^{n \times d}$0, perfectly tailored to the moment structure required by heavy-tailed analysis.

Numerical Guarantees and Comparisons

PTAS for $A \in \mathbb{R}^{n \times d}$1-LRA ($A \in \mathbb{R}^{n \times d}$2):

• For even $A \in \mathbb{R}^{n \times d}$3, achieves $A \in \mathbb{R}^{n \times d}$4 multiplicative approximation in polynomial time for fixed $A \in \mathbb{R}^{n \times d}$5.
• Relaxes prior barriers: previous $A \in \mathbb{R}^{n \times d}$6 approximations, bicriteria solutions with rank $A \in \mathbb{R}^{n \times d}$7, or additive error only.

Additive $A \in \mathbb{R}^{n \times d}$8 Norms:

• Achieves additive error $A \in \mathbb{R}^{n \times d}$9, where this term aligns with natural upper bounds by Hölder and can be tight for "well-spread" $k$0.
• Outperforms prior additive approximations—previous work [brandao2015replacing] could suffer polynomial blow-up in the error when row-norms are balanced.

These bounds are essentially sharp up to small polynomial factors and parity constraints.

Theoretical and Practical Implications

• Low-Rank Approximation: Establishes that the computational barrier for entrywise $k$1-approximation with uniform error is not fundamentally harder for $k$2 as long as $k$3 is even, modulo the assumption on bit complexity.
• Matrix Norm Approximations: Makes feasible the computation of matrix hypercontractive norms in settings crucial for small-set expansion, quantum information, and high-dimensional statistics—where exact computation is infeasible and even constant-factor approximation refutes major hypotheses.
• Convex Relaxations in Continuous Optimization: Demonstrates the full power of convex hierarchies (Sherali-Adams, SoS) for strictly continuous problems, providing a practical blueprint for similar problems with heavy-tailed noise, robust statistics, or robust optimization needs.
• Pinning Lemma Generality: The bridge between mirror descent and pinning lemmas will have relevance far outside algorithm design, including probabilistic analysis on heavy-tailed data and measure decomposition in probability theory.

Future Directions

• Extension of the PTAS to all (possibly odd) $k$4—currently the technical analysis for odd $k$5 remains open.
• Reducing dependence on $k$6 and $k$7 in the runtime (currently exponential in $k$8).
• Exploration of even finer-grained hierarchy-based relaxations for other high-dimensional, continuous, heavy-tailed optimization problems.
• Application of the "mirror descent pinning" technique in probabilistic graphical models, high-dimensional inference, and other domains involving complex dependencies.

Conclusion

This work settles a key open problem in low-rank approximation for entrywise $k$9 errors with $B$0 and provides new, practically relevant algorithms for approximating matrix $B$1 norms in regimes previously deemed intractable. The methodological innovation—pinning lemmas derived via mirror descent—provides a general template for reasoning about global correlation decay under weak moment assumptions, pushing forward both the algorithmic and analytic frontiers related to structured random variables and convex hierarchies in continuous optimization (2604.22699).