--- title: Weighted Nuclear Norm Minimization url: https://www.emergentmind.com/topics/weighted-nuclear-norm-minimization type: topic --- # Weighted Nuclear Norm Minimization Weighted nuclear norm minimization (WNNM) is a core framework for low-rank matrix recovery, matrix completion, denoising, system identification, and related problems. By assigning distinct, typically nonnegative weights to the singular values of a matrix, WNNM generalizes standard nuclear norm regularization, enabling more flexible penalization of rank components. This promotes less bias in large singular directions, more aggressive attenuation or removal of small singular values (i.e., noise), and—under appropriate weighting—retains desirable theoretical and computational properties such as convexity and closed-form shrinkage solutions. Weighted nuclear norm minimization has been extended to nonconvex settings, multi-band or multi-channel data, tensor recovery, and structured measurement and prior subspace models, and forms the basis of state-of-the-art algorithms across inverse problems and machine learning. ## 1. Mathematical Foundations and Formal Problem Statement Let \( X \in \mathbb{R}^{m \times n} \) have singular values \( \sigma_1(X) \geq \dots \geq \sigma_{\min(m,n)}(X) \geq 0 \). The (unweighted) nuclear norm is \( \|X\|_* = \sum_i \sigma_i(X) \), often used as a convex surrogate for \( \operatorname{rank}(X) \). The weighted nuclear norm is defined as \[ \|X\|_{w, *} = \sum_{i=1}^{\min(m,n)} w_i \, \sigma_i(X), \] with weights \( w_i \geq 0 \). For \( w_i = 1 \) this reduces to the usual nuclear norm. The prototypical WNNM problem, with quadratic data-fidelity, is \[ \min_X \tfrac{\mu}{2} \|X - Y\|_F^2 + \|X\|_{w, *}, \] where \( Y \) is the observation matrix and \( \mu > 0 \) adjusts fidelity versus regularization [1501.01372, 1510.06895]. Weighted nuclear norm minimization extends naturally to constrained problems (e.g., matrix completion, system identification) and to variants where the weights depend on prior knowledge, sampling structure, or principal angle information [1612.01720, 2005.10878]. Extensions to nonconvex spectral regularizers lead to penalties of the form \[ \|X\|_{w, p}^p = \sum_{i=1}^{r} w_i \sigma_i(X)^p, \quad 0 < p \leq 1, \] where \( p = 1 \) recovers WNNM, and \( p < 1 \) yields strictly nonconvex variants that better approximate the rank functional [1512.01003, 2406.15713]. ## 2. Algorithmic Structure and Closed-Form Solutions For nonnegative, non-descending weights (\( w_1 \leq w_2 \leq \dots \)), WNNM admits a closed-form solution via weighted singular value thresholding (WSVT). For \( Y = U \Sigma V^T \) (SVD), the solution to \[ \min_X \tfrac{\mu}{2} \|X - Y\|_F^2 + \|X\|_{w, *} \] is \[ X^* = U \, \operatorname{diag} \bigl( \max\{\sigma_i(Y) - w_i/\mu, 0\} \bigr) \, V^T. \] This thresholding is separable over the singular modes and, for valid weights, preserves the singular value ordering [1501.01372, 1405.6012]. For weighted Schatten \( p \)-norms, the per-singular-value subproblem becomes nonconvex for \( p < 1 \), but global minimizers can be obtained via generalized soft-thresholding (GST). Specifically, for \[ \min_{x \geq 0} \tfrac{1}{2} (x - s)^2 + \lambda w x^p, \] there is a unique minimizer computable by fixed-point iteration if \( 0 < p < 1 \), with explicit thresholds for sparsity induction [1512.01003, 2406.15713, 1901.04206]. Efficient block coordinate or ADMM solvers exist for structured formulations, including tensor decompositions and multi-channel extensions [1705.09912, 1707.07976]. ## 3. Weight Design, Convexity, and Theoretical Guarantees Convexity of the weighted nuclear norm is guaranteed when the weights are nonnegative and non-descending (\( w_1 \leq \dots \leq w_r \)), permitting the full convex analysis toolkit to be applied [1609.05944, 1405.6012]. Under these conditions, key properties hold: - \( \|X\|_{w,*} \) is a norm (unitarily invariant). - The descent cone theory and statistical dimension techniques from compressed sensing apply, predicting sharp phase transitions for exact recovery under random measurements. - There is a unique global minimizer for convex WNNM problems under standard sampling conditions. Adaptive and data-driven weight schemes are core to the empirical and theoretical success of WNNM frameworks. Noise-aware weights leverage observed or estimated singular values, typically penalizing small singular values (presumed noise) more strongly, e.g., \[ w_i = \frac{c}{\sigma_i(Y) + \varepsilon}, \] with \( c \), \( \varepsilon > 0 \) and potentially local adaptation over patch groups [1501.01372, 1702.04463, 1608.04517]. Advanced formulations use weights informed by side information or prior subspaces. For example, in matrix completion with prior subspace knowledge, left and right nuclear norm weights encode alignment confidence via \[ Q_{\widetilde{U}_r, \lambda} = \lambda P_{\widetilde{U}_r} + P_{\widetilde{U}_r^\perp}, \] with \( \lambda \ll 1 \) when prior is strong [1612.01720, 2005.10878]. Multi-weight nuclear norm minimization further extends this to allow distinct weights per principal angle, yielding strictly weaker restricted isometry requirements for recovery than single-weight or unweighted formulations [2005.10878]. ## 4. Extensions: Nonconvex, Multiblock, and Structured Models Nonconvex weighted nuclear norm minimization, typically via Schatten \( p \)-quasi-norms with \( 0