Tensor Nuclear Norm Overview

• Tensor nuclear norm is a generalization of the matrix nuclear norm defined via unit rank-one tensor decompositions and serves as the dual of the tensor spectral norm.
• The t-product based tensor nuclear norm offers an exact convex envelope of tensor average rank, underpinning recovery theory in robust PCA and tensor completion.
• Variants such as p-TNN, truncated TNN, and framelet-based TNN refine low-rank approximations, targeting specific data geometries and improving computational efficiency.

Tensor nuclear norm denotes several extensions of the matrix nuclear norm to multiway arrays. In the generic higher-order setting, it is the dual of the tensor spectral norm and can be written as the minimum total weight in a decomposition into unit rank-one tensors (Friedland et al., 2014). In the third-order $t$-SVD literature, the term often refers to the norm induced by the tensor–tensor product, defined through a block-circulant embedding or, equivalently, by Fourier-domain frontal slices; within that framework it is the convex envelope of tensor average rank on the unit ball of the tensor spectral norm (Lu et al., 2018). The literature also contains several other tensor nuclear-norm constructions and surrogates, including truncated, transform-based, nonlinear, tensor-ring, and semidefinite-relaxation variants, reflecting the fact that low-rank structure in tensors is model-dependent (Xue et al., 2019).

1. General formulations and relation to matrices

For an order-$d$ tensor $T\in\mathbb R^{n_1\times\cdots\times n_d}$, one standard definition is

$$\|T\|_*=\min\Bigl\{\sum_{i=1}^r|\lambda_i|:\;T=\sum_{i=1}^r\lambda_i\,(x_1^i\otimes\cdots\otimes x_d^i),\ \|x_k^i\|_2=1\Bigr\},$$

with dual spectral norm

$$\|T\|_\sigma=\max_{\|x_k\|_2=1}\langle T,x_1\otimes\cdots\otimes x_d\rangle.$$

When $d=2$, this reduces exactly to the matrix nuclear norm $\sum_j \sigma_j(A)$ (Friedland et al., 2014). Qi et al. state the same decomposition-based definition for general tensors and emphasize the duality $$\| \mathcal A\|_*=\max\{\langle \mathcal A,\mathcal X\rangle:\|\mathcal X\|_s\le 1\}$$ (Qi et al., 2019).

This formulation preserves several matrix-like features, but not all. Every tensor has a nuclear norm attaining decomposition, and every symmetric tensor has a symmetric nuclear norm attaining decomposition; for symmetric tensors, the symmetric nuclear norm equals the nuclear norm (Friedland et al., 2014). Nie further develops moment-SOS and Lasserre-relaxation machinery for computing symmetric tensor nuclear norms and extracting symmetric nuclear decompositions in moderate-scale cases (Nie, 2016). At the same time, higher-order behavior departs sharply from matrices: for $d\ge 3$, there exist real tensors whose real and complex nuclear norms differ, and exact computation is NP-hard in several senses (Friedland et al., 2014).

A recurring source of ambiguity is that the literature contains several definitions of tensor nuclear norm for third-order tensors, especially in completion and recovery. The $t$-SVD-based definition, the sum of unfolding nuclear norms, tensor-ring nuclear norms, and semidefinite $d$0-norm relaxations all target low-rankness, but they act on different tensor models and induce different optimization geometries (Xue et al., 2017).

2. The $d$1-product-based tensor nuclear norm

The construction introduced in tensor robust PCA is built on the tensor–tensor product. For $d$2 and $d$3,

$d$4

where $d$5 is the $d$6 block-circulant matrix built from the frontal slices of $d$7. Equivalently, one may FFT each tensor along the third dimension, multiply the resulting block-diagonal matrices slice-by-slice, and then inverse-FFT (Lu et al., 2018).

This algebra supports a $d$8-SVD

$d$9

with $T\in\mathbb R^{n_1\times\cdots\times n_d}$0 orthogonal under the $T\in\mathbb R^{n_1\times\cdots\times n_d}$1-product and $T\in\mathbb R^{n_1\times\cdots\times n_d}$2 $T\in\mathbb R^{n_1\times\cdots\times n_d}$3-diagonal. If $T\in\mathbb R^{n_1\times\cdots\times n_d}$4, the tensor spectral norm and tensor nuclear norm are defined by

$T\in\mathbb R^{n_1\times\cdots\times n_d}$5

In the Fourier-domain viewpoint, if $T\in\mathbb R^{n_1\times\cdots\times n_d}$6 has frontal slices $T\in\mathbb R^{n_1\times\cdots\times n_d}$7, then

$T\in\mathbb R^{n_1\times\cdots\times n_d}$8

These definitions extend the matrix case exactly when $T\in\mathbb R^{n_1\times\cdots\times n_d}$9 (Lu et al., 2018).

The same framework defines the tensor average rank

$$\|T\|_*=\min\Bigl\{\sum_{i=1}^r|\lambda_i|:\;T=\sum_{i=1}^r\lambda_i\,(x_1^i\otimes\cdots\otimes x_d^i),\ \|x_k^i\|_2=1\Bigr\},$$0

In related $$\|T\|_*=\min\Bigl\{\sum_{i=1}^r|\lambda_i|:\;T=\sum_{i=1}^r\lambda_i\,(x_1^i\otimes\cdots\otimes x_d^i),\ \|x_k^i\|_2=1\Bigr\},$$1-SVD formulations, one also encounters tubal rank, defined as the number of nonzero singular tubes or, equivalently, $$\|T\|_*=\min\Bigl\{\sum_{i=1}^r|\lambda_i|:\;T=\sum_{i=1}^r\lambda_i\,(x_1^i\otimes\cdots\otimes x_d^i),\ \|x_k^i\|_2=1\Bigr\},$$2 after FFT along the third mode (Zhang et al., 2022). The distinction matters: average-rank statements and tubal-rank statements are not interchangeable unless a paper states the relevant equivalence or reduction.

3. Convex-envelope geometry and recovery theory

A central theorem of the $$\|T\|_*=\min\Bigl\{\sum_{i=1}^r|\lambda_i|:\;T=\sum_{i=1}^r\lambda_i\,(x_1^i\otimes\cdots\otimes x_d^i),\ \|x_k^i\|_2=1\Bigr\},$$3-product-based construction is that on the unit ball

$$\|T\|_*=\min\Bigl\{\sum_{i=1}^r|\lambda_i|:\;T=\sum_{i=1}^r\lambda_i\,(x_1^i\otimes\cdots\otimes x_d^i),\ \|x_k^i\|_2=1\Bigr\},$$4

the convex envelope of $$\|T\|_*=\min\Bigl\{\sum_{i=1}^r|\lambda_i|:\;T=\sum_{i=1}^r\lambda_i\,(x_1^i\otimes\cdots\otimes x_d^i),\ \|x_k^i\|_2=1\Bigr\},$$5 is exactly $$\|T\|_*=\min\Bigl\{\sum_{i=1}^r|\lambda_i|:\;T=\sum_{i=1}^r\lambda_i\,(x_1^i\otimes\cdots\otimes x_d^i),\ \|x_k^i\|_2=1\Bigr\},$$6 (Lu et al., 2018). The proof proceeds through convex conjugates. Writing $$\|T\|_*=\min\Bigl\{\sum_{i=1}^r|\lambda_i|:\;T=\sum_{i=1}^r\lambda_i\,(x_1^i\otimes\cdots\otimes x_d^i),\ \|x_k^i\|_2=1\Bigr\},$$7, the conjugate $$\|T\|_*=\min\Bigl\{\sum_{i=1}^r|\lambda_i|:\;T=\sum_{i=1}^r\lambda_i\,(x_1^i\otimes\cdots\otimes x_d^i),\ \|x_k^i\|_2=1\Bigr\},$$8 can be expressed through the singular values of the block-circulant embedding, and von Neumann’s trace inequality yields the optimal alignment argument. The biconjugate $$\|T\|_*=\min\Bigl\{\sum_{i=1}^r|\lambda_i|:\;T=\sum_{i=1}^r\lambda_i\,(x_1^i\otimes\cdots\otimes x_d^i),\ \|x_k^i\|_2=1\Bigr\},$$9, restricted to $$\|T\|_\sigma=\max_{\|x_k\|_2=1}\langle T,x_1\otimes\cdots\otimes x_d\rangle.$$0, then reduces to

$$\|T\|_\sigma=\max_{\|x_k\|_2=1}\langle T,x_1\otimes\cdots\otimes x_d\rangle.$$1

This reproduces the matrix relation between nuclear norm, spectral norm, and rank in a tensor algebra induced by the $$\|T\|_\sigma=\max_{\|x_k\|_2=1}\langle T,x_1\otimes\cdots\otimes x_d\rangle.$$2-product (Lu et al., 2018).

That convex-envelope statement is specific to the $$\|T\|_\sigma=\max_{\|x_k\|_2=1}\langle T,x_1\otimes\cdots\otimes x_d\rangle.$$3-product model. The same paper stresses that, unlike just summing matrix nuclear norms of all slices, the new tensor nuclear norm averages over the block-circulant embedding, ensuring the convex-envelope property holds. In this sense, the $$\|T\|_\sigma=\max_{\|x_k\|_2=1}\langle T,x_1\otimes\cdots\otimes x_d\rangle.$$4-SVD-based TNN is not merely a heuristic slice regularizer but a precise convex relaxation of tensor average rank in the associated non-commutative algebra (Lu et al., 2018).

Recovery theory in the same framework extends beyond robust PCA. By choosing an atomic set adapted to tubal rank, Zhang and Aeron show that TNN is a special atomic norm and that exact recovery from Gaussian measurements of a tensor of size $$\|T\|_\sigma=\max_{\|x_k\|_2=1}\langle T,x_1\otimes\cdots\otimes x_d\rangle.$$5 and tubal rank $$\|T\|_\sigma=\max_{\|x_k\|_2=1}\langle T,x_1\otimes\cdots\otimes x_d\rangle.$$6 requires

$$\|T\|_\sigma=\max_{\|x_k\|_2=1}\langle T,x_1\otimes\cdots\otimes x_d\rangle.$$7

which is order optimal relative to the degrees of freedom $$\|T\|_\sigma=\max_{\|x_k\|_2=1}\langle T,x_1\otimes\cdots\otimes x_d\rangle.$$8 (Lu et al., 2018). The same work gives tensor-completion guarantees under uniform random sampling and incoherence assumptions, with sample complexity $$\|T\|_\sigma=\max_{\|x_k\|_2=1}\langle T,x_1\otimes\cdots\otimes x_d\rangle.$$9 (Lu et al., 2018).

In tensor robust PCA, the $d=2$0-product-based TNN leads to a convex program that exactly recovers low-rank and sparse components under incoherence and sparsity assumptions, paralleling matrix PCP theory. Matrix RPCA appears as a special case, and the reported applications include image recovery and background modeling (Lu et al., 2018).

4. Computation, proximal mappings, and algorithmic use

In the $d=2$1-SVD setting, computing the TNN and its proximal operator is FFT-centric. One performs an FFT along mode 3, computes matrix SVDs on the frontal slices in the transform domain, applies slice-wise singular-value thresholding, and then returns by inverse FFT. For the proximal map of $d=2$2, the tensor singular value thresholding (TSVT) procedure consists of: FFT, $d=2$3 slice SVDs, soft-thresholding of singular values, and inverse FFT (Zhang et al., 2022).

The per-iteration complexity reported for TSVT is one FFT and one inverse FFT of size $d=2$4, together with $d=2$5 independent SVDs of size $d=2$6 (Zhang et al., 2022). This decomposition is the basic computational reason why ADMM and proximal-gradient methods are practical for large third-order tensors in imaging and completion.

A direct application is dynamic cardiac MRI reconstruction. The TMNN model combines the $d=2$7-SVD-based TNN with the Casorati matrix nuclear norm: $d=2$8 The motivation is explicit: TNN exploits spatial structure of the dynamic MR data, while the Casorati matrix nuclear norm exploits temporal correlation. The resulting ADMM solver admits a fast Cartesian-sampling variant, and the reported experiments show up to $d=2$9 dB SNR gain over plain MNN together with an $\sum_j \sigma_j(A)$0 run-time reduction for the fast $\sum_j \sigma_j(A)$1-space implementation (Zhang et al., 2022).

The same study also states a useful structural caveat: in general,

$\sum_j \sigma_j(A)$2

Accordingly, TNN and Casorati nuclear norms should be regarded as complementary regularizers rather than interchangeable ones (Zhang et al., 2022).

5. Variants, tighter surrogates, and transform-induced extensions

Several later works keep the $\sum_j \sigma_j(A)$3-SVD backbone but modify the singular-value penalty or the transform domain to approximate tensor rank more tightly.

Tensor $\sum_j \sigma_j(A)$4-shrinkage nuclear norm. The $\sum_j \sigma_j(A)$5-TNN replaces soft-thresholding by the scalar $\sum_j \sigma_j(A)$6-shrinkage operator

$\sum_j \sigma_j(A)$7

applied to the diagonal entries of the $\sum_j \sigma_j(A)$8-SVD core in the Fourier domain. The resulting functional is positive, unitary-invariant, and non-convex for $\sum_j \sigma_j(A)$9, and it satisfies

$$\| \mathcal A\|_*=\max\{\langle \mathcal A,\mathcal X\rangle:\|\mathcal X\|_s\le 1\}$$0

The paper states that $$\| \mathcal A\|_*=\max\{\langle \mathcal A,\mathcal X\rangle:\|\mathcal X\|_s\le 1\}$$1-TNN is a better approximation of tensor average rank than TNN when $$\| \mathcal A\|_*=\max\{\langle \mathcal A,\mathcal X\rangle:\|\mathcal X\|_s\le 1\}$$2, gives a statistical recovery-error bound for low-rank tensor completion, and analyzes an ADMM solver with adaptive momentum whose convergence rate is $$\| \mathcal A\|_*=\max\{\langle \mathcal A,\mathcal X\rangle:\|\mathcal X\|_s\le 1\}$$3 under the smoothness assumption (Liu et al., 2019).

Tensor truncated nuclear norm. T-TNN generalizes matrix truncated nuclear norm to the $$\| \mathcal A\|_*=\max\{\langle \mathcal A,\mathcal X\rangle:\|\mathcal X\|_s\le 1\}$$4-SVD setting by subtracting the contribution of the leading $$\| \mathcal A\|_*=\max\{\langle \mathcal A,\mathcal X\rangle:\|\mathcal X\|_s\le 1\}$$5 singular values. In one formulation,

$$\| \mathcal A\|_*=\max\{\langle \mathcal A,\mathcal X\rangle:\|\mathcal X\|_s\le 1\}$$6

so only the tail singular values of the first Fourier frontal slice are penalized. The stated motivation is that truncation avoids over-shrinkage of dominant components and more closely approximates tubal rank than the plain T-NN. The ADMM and APGL schemes require one matrix SVD of size $$\| \mathcal A\|_*=\max\{\langle \mathcal A,\mathcal X\rangle:\|\mathcal X\|_s\le 1\}$$7 per iteration, and the reported image-completion experiments show a $$\| \mathcal A\|_*=\max\{\langle \mathcal A,\mathcal X\rangle:\|\mathcal X\|_s\le 1\}$$8–$$\| \mathcal A\|_*=\max\{\langle \mathcal A,\mathcal X\rangle:\|\mathcal X\|_s\le 1\}$$9 speedup over several baselines (Xue et al., 2017).

Framelet-based tensor nuclear norm. F-TNN replaces the DFT along each tube by a redundant framelet transform $d\ge 3$0 satisfying $d\ge 3$1. The norm is then defined as the sum of matrix nuclear norms of the framelet-transformed frontal slices. Because of framelet-basis redundancy, the representation of each tube is sparsely represented, and the paper reports that on MRI, video, and multispectral data the average truncated frontal-slice rank can drop by $d\ge 3$2–$d\ge 3$3. The completion model is convex, global minimizers can be obtained, and empirical gains of $d\ge 3$4–$d\ge 3$5 dB in PSNR over several baselines are reported (Jiang et al., 2019).

Nonlinear transform induced tensor nuclear norm. NTTNN augments a semi-orthogonal linear transform along mode 3 with an element-wise nonlinear activation $d\ge 3$6, giving

$d\ge 3$7

The paper argues that the nonlinearity can shrink small entries and emphasize dominant modes in the transformed frontal slices, and it solves the resulting nonlinear, nonconvex completion model with proximal alternating minimization under a Kurdyka–Łojasiewicz analysis. Reported gains are $d\ge 3$8–$d\ge 3$9 dB PSNR over TNN and $t$0–$t$1 dB over learned-transform TNNs on hyperspectral images, multispectral images, and videos (Li et al., 2021).

Multimode nonlinear transform-based TNN. MNT-TNN extends single-mode TTNN to a multimode setting using a face-wise mode-$t$2 transform $t$3, a mode-$t$4 transform $t$5, a mode-3 transform $t$6, and an element-wise nonlinearity $t$7. The paper formulates a PAM algorithm with sufficient-decrease and relative-error guarantees under the KŁ framework, and introduces ATTNN chains that first use linear TTNN variants and then nonlinear TTNN variants. At very high missing rates, ATTNNs are reported to deliver $t$8–$t$9 further MAPE/RMSE improvements over standalone MNT-TNN in spatiotemporal traffic imputation (Lu et al., 29 Mar 2025).

Tensor-ring nuclear norm. A different line of work defines

$d$00

where $d$01 are circular unfoldings. The stated theoretical link is $d$02 when $d$03 has TR ranks $d$04. The resulting completion model is convex and is solved by ADMM; in stripe-missing image and video completion it is reported to outperform several conventional tensor-completion methods (Yu et al., 2019).

6. Higher-order structure, bounds, relaxations, and broader roles

For generic higher-order tensors, structural and computational questions remain difficult. Friedland and Lim show that nuclear norm depends on the base field for tensors of order at least three, that the nuclear norm unit ball and its weak-membership problem are computationally hard, and that computing spectral or nuclear norm is NP-hard for several restricted tensor classes (Friedland et al., 2014). Qi et al. add that the $d$05-norm, Frobenius norm, and nuclear norm are tensor norms in the sense of vector-norm axioms plus submultiplicativity under outer products, whereas the infinity norm and spectral norm are not tensor norms (Qi et al., 2019).

Despite the intractability of exact computation, there are computable bounds. For an $d$06-tensor $d$07, the nuclear norm of every matrix flattening $d$08 is a lower bound for $d$09, and for 3-tensors one has

$d$10

with both bounds sharp when $d$11 (Hu, 2014). For third-order tensors, contraction to positive semidefinite biquadratic tensors gives additional lower bounds: the square roots of the nuclear norms of the three contracted biquadratic tensors are lower bounds of the tensor nuclear norm (Qi et al., 2019).

Semidefinite relaxations provide another route. The $d$12-norms are defined through theta bodies of polynomial ideals generated by second-order minors of tensor matricizations, with the property that in the matrix case they reduce to the nuclear norm, while for order $d$13 they give new norms. The unit-$d$14-norm balls converge asymptotically to the unit tensor nuclear norm ball, and computing $d$15-norms or minimizing them under affine constraints reduces to semidefinite programming (Rauhut et al., 2015).

Recent work on decomposability and subdifferentials shows that the tensor nuclear norm admits full decomposability over specific subspaces such as

$d$16

and identifies the largest possible subspaces allowing exact additivity. The same work derives new inclusions for the subdifferential and uses them to establish the statistical performance of tensor robust PCA for tensors of arbitrary order, described there as the first such result in that generality (Guan et al., 6 Oct 2025).

The tensor nuclear norm also appears outside completion and denoising. In bilinear complexity, a bilinear operator $d$17 can be identified with a 3-tensor, and its nuclear norm

$d$18

equals the minimum growth factor of any bilinear algorithm for $d$19. The forward-error bound proved in that setting ties numerical accuracy directly to $d$20, yielding a tensorial notion of bilinear stability that is invariant under orthogonal change of coordinates (Dai et al., 2022).

Taken together, these results show that “tensor nuclear norm” is not a single universally adopted object but a family of norms and norm-like surrogates attached to distinct tensor models. The decomposition-based higher-order norm provides the broad convex analogue of matrix nuclear norm; the $d$21-SVD-based norm provides an exact convex envelope of tensor average rank in the $d$22-product algebra; and later transform-based or truncated variants modify the penalty to target specific data geometries or rank surrogates more tightly (Friedland et al., 2014).