Nuclear Norm of Higher-Order Tensors (1410.6072v3)

Abstract: We establish several mathematical and computational properties of the nuclear norm for higher-order tensors. We show that like tensor rank, tensor nuclear norm is dependent on the choice of base field --- the value of the nuclear norm of a real 3-tensor depends on whether we regard it as a real 3-tensor or a complex 3-tensor with real entries. We show that every tensor has a nuclear norm attaining decomposition and every symmetric tensor has a symmetric nuclear norm attaining decomposition. There is a corresponding notion of nuclear rank that, unlike tensor rank, is upper semicontinuous. We establish an analogue of Banach's theorem for tensor spectral norm and Comon's conjecture for tensor rank --- for a symmetric tensor, its symmetric nuclear norm always equals its nuclear norm. We show that computing tensor nuclear norm is NP-hard in several sense. Deciding weak membership in the nuclear norm unit ball of 3-tensors is NP-hard, as is finding an $\varepsilon$-approximation of nuclear norm for 3-tensors. In addition, the problem of computing spectral or nuclear norm of a 4-tensor is NP-hard, even if we restrict the 4-tensor to be bi-Hermitian, bisymmetric, positive semidefinite, nonnegative valued, or all of the above. We discuss some simple polynomial-time approximation bounds. As an aside, we show that the nuclear $(p,q)$-norm of a matrix is NP-hard in general but can be computed in polynomial-time if $p=1$, $q = 1$, or $p=q=2$, with closed-form expressions for the nuclear $(1,q)$- and $(p,1)$-norms.

• The paper demonstrates that the nuclear norm of tensors is influenced by the choice of base field, establishing distinct decomposition properties for symmetric and non-symmetric tensors.
• It reveals that computing the nuclear norm for higher-order tensors, particularly 4-tensors, is NP-hard even under structured conditions, challenging standard approximation methods.
• The study introduces the concept of an upper semicontinuous nuclear rank, offering a foundation for more stable tensor computations and inspiring future algorithmic research.

An Expert Analysis of "Nuclear Norm of Higher-Order Tensors"

The paper "Nuclear Norm of Higher-Order Tensors" by Shmuel Friedland and Lek-Heng Lim offers a thorough examination of the nuclear norms in the context of higher-order tensors, presenting several critical properties and computational challenges associated with these norms. The exploration of nuclear norms in tensors is an extension of concepts traditionally applied to matrices, but with additional complexities given the multidimensional nature of tensors.

Theoretical Foundations

The authors establish that the nuclear norm, like tensor rank, is influenced by the choice of base field, whether real or complex. This is significant because it highlights the dependency of tensor properties on the underlying field, a characteristic not found in matrix norms. The paper asserts that every tensor possesses a nuclear norm attaining decomposition, and similarly, symmetric tensors have symmetric nuclear norm decompositions. This establishes a parallel to Comon's conjecture for tensor rank, particularly for symmetric tensors where the symmetric nuclear norm aligns with the nuclear norm.

A notable contribution is the definition of nuclear rank, which unlike traditional tensor rank, is upper semicontinuous. This property mitigates issues related to the best rank-r approximation problem, a persistent challenge in tensor computations.

Computational Complexity

The paper's authors delve into computational difficulties, revealing that the calculation of tensor nuclear norms is NP-hard across several dimensions and tensor types. Specifically, they identify the NP-hard nature of computing spectral or nuclear norms for 4-tensors, even under constraints such as bi-Hermitian, bisymmetric, or positive semidefinite conditions. Moreover, they provide significant insight into approximation problems, delineating polynomial-time bounds for spectral and nuclear norms.

Implications and Future Directions

The implications of these findings are profound for both theoretical research and practical applications in AI and machine learning. The upper semicontinuity of nuclear rank offers a potential avenue for more stable computations in tensor analysis. However, the inherent NP-hard nature of these computations suggests a need for further research into efficient approximation algorithms. As tensor applications continue to expand in fields such as signal processing and quantum computing, understanding these computational complexities will be pivotal.

Conclusion

This paper provides substantial advancements in the understanding of tensor nuclear norms and their computational intricacies. The interplay between the properties of tensors and the choice of base field opens new lines of inquiry regarding tensor algebra. Future research might focus on developing algorithms that can either circumvent or minimize the NP-hard barriers identified by Friedland and Lim. Overall, the paper represents a valuable contribution to the paper of higher-order tensor norms, providing a foundation for future explorations in the computational aspects of tensor analysis.