Schatten-p Quasi-Norm in Low-Rank Optimization

• Schatten-p quasi-norm is a nonconvex norm defined on singular values that interpolates between the nuclear norm and the rank function.
• It enables scalable optimization via factorization-based formulations and iterative reweighted schemes, avoiding full SVD computations.
• Applications include low-rank matrix/tensor completion, robust PCA, and quantum information, offering improved recovery guarantees and performance.

The Schatten-p quasi-norm is a central tool in modern low-rank modeling and nonconvex optimization, bridging spectral convex relaxations (nuclear norm) and the combinatorial rank function. Formally, for a matrix $X\in\mathbb{R}^{m\times n}$ with singular values $\sigma_1(X)\geq\sigma_2(X)\geq\cdots\geq0$, the Schatten-p quasi-norm for $0

$$\|X\|_{S_p} := \left(\sum_{i=1}^{\min(m,n)} \sigma_i(X)^p\right)^{1/p}$$

It is a nonconvex, non-smooth, quasi-norm for $0$p=1$, convex) and the rank function ($p\rightarrow0$), and is equipped with a weakened subadditivity property. Recent years have seen the emergence of scalable algorithms, refined theoretical guarantees, and generalizations to tensors and operator algebras.

1. Definitions, Properties, and Motivations

The Schatten-p quasi-norm $\|X\|_{S_p}$ generalizes the $\ell_p$ quasi-norm on vectors to the singular value vector of a matrix. It is nonconvex for $p<1$, homogeneous, unitarily invariant, and satisfies the quasi-triangle property: $\sigma_1(X)\geq\sigma_2(X)\geq\cdots\geq0$0 No universal $\sigma_1(X)\geq\sigma_2(X)\geq\cdots\geq0$1 exists with $\sigma_1(X)\geq\sigma_2(X)\geq\cdots\geq0$2 for $\sigma_1(X)\geq\sigma_2(X)\geq\cdots\geq0$3 (Yue et al., 2012, Sobolev, 2013). The quasi-norm is a much tighter relaxation of rank than the nuclear norm. As $\sigma_1(X)\geq\sigma_2(X)\geq\cdots\geq0$4, one recovers the nuclear norm; as $\sigma_1(X)\geq\sigma_2(X)\geq\cdots\geq0$5, the function approximates rank: $\sigma_1(X)\geq\sigma_2(X)\geq\cdots\geq0$6 Minimizing the rank function directly is NP-hard; the Schatten-p quasi-norm offers a family of heuristics that interpolate between tractability and approximation tightness (Shang et al., 2018, Malek-Mohammadi et al., 2014, Shang et al., 2016). In the operator setting, Schatten–von Neumann classes $\sigma_1(X)\geq\sigma_2(X)\geq\cdots\geq0$7 are defined for compact operators via the same singular value decay, and for $\sigma_1(X)\geq\sigma_2(X)\geq\cdots\geq0$8, $\sigma_1(X)\geq\sigma_2(X)\geq\cdots\geq0$9 is a quasi-Banach space (Sobolev, 2013).

2. Variational and Factorization-Based Formulations

For general $0

• Product of Factor Norms (Shang et al., 2016, Xu et al., 2016):

$0

• Weighted Sum Representation:

$0

• Special Cases:

For $0Shang et al., 2018, Shang et al., 2016). For $0Shang et al., 2016).

These factorizations eliminate full SVDs in iterative optimization and enable scalable algorithms for large-scale problems. For general $0Shang et al., 2016, Xu et al., 2016).

For tensors, analogous variational formulations based on CP decompositions (sums of rank-1 outer products) are established. E.g., for order-$0$$\|X\|_{S_p} := \left(\sum_{i=1}^{\min(m,n)} \sigma_i(X)^p\right)^{1/p}$$0, the Schatten-p quasi-norm takes the form (Cheng et al., 27 Jun 2025, Fan et al., 2020): $$\|X\|_{S_p} := \left(\sum_{i=1}^{\min(m,n)} \sigma_i(X)^p\right)^{1/p}$$1

3. Optimization Algorithms and Scalability

Proximal- and Alternating-Minimization-Based Methods

Direct minimization of $$\|X\|_{S_p} := \left(\sum_{i=1}^{\min(m,n)} \sigma_i(X)^p\right)^{1/p}$$2 is prohibitive due to SVD computation per iteration and nonconvexity. Modern approaches exploit factorized surrogates:

• Alternating Linearized Minimization (PALM, LADM, APALM) (Shang et al., 2018, Shang et al., 2016, Xu et al., 2016): Alternately update matrix factors (e.g., $$\|X\|_{S_p} := \left(\sum_{i=1}^{\min(m,n)} \sigma_i(X)^p\right)^{1/p}$$3, $$\|X\|_{S_p} := \left(\sum_{i=1}^{\min(m,n)} \sigma_i(X)^p\right)^{1/p}$$4 for bi-trace/bi-nuclear; $$\|X\|_{S_p} := \left(\sum_{i=1}^{\min(m,n)} \sigma_i(X)^p\right)^{1/p}$$5, $$\|X\|_{S_p} := \left(\sum_{i=1}^{\min(m,n)} \sigma_i(X)^p\right)^{1/p}$$6, $$\|X\|_{S_p} := \left(\sum_{i=1}^{\min(m,n)} \sigma_i(X)^p\right)^{1/p}$$7 for tri-trace) using block-wise proximal or gradient steps, where each subproblem has a closed-form (e.g., singular value soft-thresholding for trace-norm blocks). This results in per-iteration cost scaling as $$\|X\|_{S_p} := \left(\sum_{i=1}^{\min(m,n)} \sigma_i(X)^p\right)^{1/p}$$8 versus $$\|X\|_{S_p} := \left(\sum_{i=1}^{\min(m,n)} \sigma_i(X)^p\right)^{1/p}$$9 for full SVD.
• Iterative Reweighted Schemes (Lu et al., 2014, Wang et al., 2016): Solve a sequence of weighted nuclear norm (or $0
• Dynamic Proximal Gradient (Shen et al., 27 Feb 2026): Utilizes Cayley transformations to sidestep repeated SVD when updating both singular values and singular vectors; convergence is established under Kurdyka–Łojasiewicz (KL) property, with sublinear rate for all $0

Algorithmic and Empirical Performance

Empirically, scalable factor methods like bi-trace and tri-trace quasi-norms are orders of magnitude faster than classical nuclear or Schatten-p quasi-norm minimization (which require repeated full SVDs). On large collaborative filtering datasets (MovieLens, Netflix) and RPCA tasks, these methods yield lower RMSE or RSE, better rank recovery, and efficient convergence for problems with matrices up to $0Shang et al., 2018). In tensor settings, similar scaling and accuracy improvements are observed for low-rank tensor completion and robust PCA (Cheng et al., 27 Jun 2025, Fan et al., 2020).

4. Theoretical Recovery and Generalization Guarantees

Sharp conditions for low-rank matrix recovery via Schatten-p minimization are established:

• Null Space Property (NSP) (Yue et al., 2012, Malek-Mohammadi et al., 2014):

$0

is necessary and sufficient for exact rank-$0

• Restricted Isometry Property (RIP) Lifting (Malek-Mohammadi et al., 2014, Yue et al., 2012): Any RIP-based $0
• Error Bounds under Noise: With restricted strong convexity (RSC) of the sampling operator, error bounds are derived for critical points of bi-trace and tri-trace-regularized minimization. E.g., in matrix completion, only $0Shang et al., 2018).

In tensor completion and robust PCA, sharper generalization and recovery bounds are achieved for smaller $p=1$0; for order-$p=1$1 tensors, $p=1$2 provides optimal error rates (Fan et al., 2020). Theoretical excess risk bounds for low-rank neural representations regularized by the Schatten-p quasi-norm are given in terms parallel to those of matrix/tensor estimation (Cheng et al., 27 Jun 2025).

5. Perturbation, Structural, and Operator-Algebraic Aspects

Singular Value Perturbation and Structural Results

A fundamental perturbation inequality states that for all $p=1$3, and $p=1$4,

$p=1$5

This result underpins NSP and RIP analyses and enables a direct lifting of vector compressed sensing theory to the nonconvex, matrix-valued setting (Yue et al., 2012, Malek-Mohammadi et al., 2014).

Schatten-p Quasi-Norms for Operators

For compact operators on Hilbert spaces, the Schatten-p quasi-norm $p=1$6 class is the natural noncommutative generalization, with operator-analytic estimates (e.g., for pseudo-differential operators) scaling as $p=1$7 for semiclassical parameter $p=1$8, generalizing classical trace-class ($p=1$9) bounds (Sobolev, 2013).

Two-Indexed Quasi-Norms and Quantum Information

The $p\rightarrow0$0-Schatten quasi-norms extend to bipartite operator spaces, with compatibility condition $p\rightarrow0$1 necessary for key structural properties such as block-diagonal consistency, unitary invariance, and quasi-triangle inequality (Kochanowski et al., 15 Apr 2026). These quasi-norms underpin completely bounded and co-quasi-norms, essential in describing quantum channel capacities and sandwiched Rényi entropies.

6. Applications: Matrix and Tensor Completion, Robust PCA, and Beyond

The Schatten-p quasi-norm framework has been widely adopted for:

• Low-Rank Matrix Completion: For large incomplete datasets (recommender systems), Schatten-p regularization yields superior error rates and lower-rank solutions than nuclear norm approaches, with empirical and theoretical recovery guarantees (Shang et al., 2018, Malek-Mohammadi et al., 2014, Shang et al., 2016).
• Robust PCA: Joint Schatten-p (for low-rank) and $p\rightarrow0$2 (for sparse corruption) nonconvex penalties align more closely with true data structure and exhibit better support and singular value recovery, with globally convergent reweighted algorithms (Wang et al., 2016).
• Multi-dimensional Data Recovery: For color image, video, and hyperspectral data processing, tensor Schatten-p quasi-norms (or variational surrogates via CP-factors) attain state-of-the-art in denoising, inpainting, and upsampling (Fan et al., 2020, Cheng et al., 27 Jun 2025).
• Implicit Neural Representations: Continuous-domain data recovery pipelines using coordinate-based MLPs with Schatten-p quasi-norm regularization on CP weights achieve substantial sparsification and excess-risk minimization (Cheng et al., 27 Jun 2025).
• Pseudo-differential Operator Analysis: Schatten quasi-norm estimates afford semiclassical and regularity bounds for PDEs and quantum mechanical systems (Sobolev, 2013).
• Quantum Information Theory: Two-indexed Schatten quasi-norms express Rényi entropies and enable multiplicativity/additivity results for quantum channels (Kochanowski et al., 15 Apr 2026).

7. Selection of $p\rightarrow0$3 and Algorithmic Guidelines

Selecting $p\rightarrow0$4 balances convexity against rank approximation:

• Smaller $p\rightarrow0$5 ($p\rightarrow0$6) yields tighter approximation to rank and greater sparsity in the singular spectrum, with empirical improvements in recovery—at the cost of increased nonconvexity and potential numerical instability.
• For tensors of order $p\rightarrow0$7, optimal theoretical generalization bounds suggest $p\rightarrow0$8 (Fan et al., 2020).
• Empirically, $p\rightarrow0$9–$\|X\|_{S_p}$0 maximizes sparsity and recovery performance in neural and tensor settings (Cheng et al., 27 Jun 2025).
• For practical optimization, use variational/factorized surrogates with as many smooth, convex blocks as possible (multi-factor, $\|X\|_{S_p}$1 for severe nonconvexity), and apply alternating minimization or reweighted schemes (Shang et al., 2018, Shang et al., 2016, Shang et al., 2016).

Performance, convergence, and critical-point guarantees for these algorithms are available via monotonicity of objective, KL property, and blockwise Lipschitz continuity. In practice, factor-based surrogates, reweighted proximal steps, or SVD-free dynamic updates enable tractable large-scale computation for any $\|X\|_{S_p}$2.