Most tensor problems are NP-hard (0911.1393v5)

Abstract: We prove that multilinear (tensor) analogues of many efficiently computable problems in numerical linear algebra are NP-hard. Our list here includes: determining the feasibility of a system of bilinear equations, deciding whether a 3-tensor possesses a given eigenvalue, singular value, or spectral norm; approximating an eigenvalue, eigenvector, singular vector, or the spectral norm; and determining the rank or best rank-1 approximation of a 3-tensor. Furthermore, we show that restricting these problems to symmetric tensors does not alleviate their NP-hardness. We also explain how deciding nonnegative definiteness of a symmetric 4-tensor is NP-hard and how computing the combinatorial hyperdeterminant of a 4-tensor is NP-, #P-, and VNP-hard. We shall argue that our results provide another view of the boundary separating the computational tractability of linear/convex problems from the intractability of nonlinear/nonconvex ones.

• The paper establishes that tensor problems, including eigenvalue, spectral norm, and rank computations, are NP-hard.
• The authors leverage reductions from NP-complete problems, such as graph 3-colorability, to rigorously demonstrate the intractability of these multilinear challenges.
• These findings highlight that conventional matrix algorithms cannot be naively extended to tensors, prompting a need for new heuristic and approximation methods.

Most Tensor Problems are NP-Hard

The paper co-authored by Christopher J. Hillar and Lek-Heng Lim, titled "Most Tensor Problems are NP-Hard", provides a thorough mathematical analysis demonstrating the computational complexity of a variety of tensor-related problems. The authors establish that multilinear analogues of numerous efficiently computable problems in numerical linear algebra are, in fact, NP-hard. This work is significant in that it extends well-understood computational problems from the field of linear to multilinear algebra, revealing substantial increases in complexity.

Overview of Results

The authors rigorously prove that numerous problems, which are computationally tractable for matrices, become NP-hard when extended to tensors. Here are some key results:

1. Bilinear Systems: Determining the feasibility of a system of bilinear equations,\ i.e., whether there exists a nontrivial solution to a set of multilinear equations, is NP-hard.
2. Eigenvalues and Eigenvectors: Deciding whether a 3-tensor possesses a given eigenvalue, singular value, or spectral norm, as well as approximating these quantities, is NP-hard.
3. Tensor Rank: Determining the rank or best rank-1 approximation of a 3-tensor is NP-hard.
4. Symmetric Tensors: Restricting these problems to symmetric tensors does not alleviate their NP-hardness. In particular, nonnegative definiteness of a symmetric 4-tensor and the computation of the combinatorial hyperdeterminant of a 4-tensor are shown to be NP-, #P-, and VNP-hard.

Detailed Insights

The main contribution of the paper is to establish the intractability of a wide range of tensor problems, which ten computational tracts are feasible for matrices.

1. Tensor Eigenvalues and Singular Values:
   • Eigenvalue Problem: The paper shows that determining whether a given scalar is an eigenvalue of a 3-tensor is NP-hard. This conclusion is reached through a reduction from the graph 3-colorability problem.
   • Singular Value Problem: Similarly, the decision problem for tensor singular values is NP-hard, involving reductions to existing combinatorial problems.
2. Spectral Norm:
   • Deciding Spectral Norm: Establishing whether a given value is the spectral norm of a tensor is NP-hard.
   • Approximation Algorithms: The paper also addresses the difficulty in approximating these quantities, proving that even approximation schemes such as PTAS and FPTAS cannot exist for tensor problems, unless P=NP.
3. Tensor Rank:
   • Rank Determination: Determining the rank of a tensor over fields like $\mathbb{Q}$ and $\mathbb{R}$ is proven NP-hard. This includes showing the complexity involved even when understanding tensor rank over different fields, demonstrating that results differ based on the field (e.g., $\mathbb{Q}$ vs. $\mathbb{R}$).
4. Symmetric Tensors:
   • Nonnegative Definiteness: Deciding whether a symmetric 4-tensor is nonnegative definite is also shown to be NP-hard.
   • Hyperdeterminants: The paper includes a discussion on hyperdeterminants, proving their evaluation is highly complex, specifically demonstrating NP-, #P-, and VNP-hardness.

Theoretical and Practical Implications

The paper highlights a clear boundary between the computational feasibility of linear problems and the intractability of their multilinear counterparts. The theoretical implications are profound, providing a deep understanding of the hardness of generalizing well-known linear algebraic problems to tensors. Practically, this work cautions against naive extensions of matrix algorithms to tensors without considering computational complexities.

Speculations on Future Developments

Given the NP-hardness of the examined tensor problems, future research in these areas may focus on:

• Approximation Algorithms: Developing approximation algorithms with guaranteed performance bounds, even though perfect solutions remain out of reach.
• Heuristic Methods: Enhancing heuristic methods and exploring their applicability to specific types of tensors or particular applications.
• Quantum and Parallel Computing: Investigating whether tensor problems can benefit significantly from the capabilities of quantum computing and parallel processing to mitigate their inherent complexities.

Conclusion

The research by Hillar and Lim plays a crucial role in understanding the computational boundaries of multilinear algebra. By demonstrating the NP-hard nature of many tensor problems, the paper sets a benchmark for future explorations in both theoretical and application-oriented multilinear algebra, emphasizing the need for novel approaches to tackle these inherently complex problems.