Moment Tensor Decomposition Overview

• Moment tensor decomposition is a set of methods that represent and compress high-order symmetric moment tensors using techniques like symmetric Tucker and CP decompositions.
• It employs low-rank approximations, iterative optimization, and implicit strategies to handle high-dimensional data without requiring full tensor formation.
• The approach unifies algebraic recovery, tensor-free computation, and physical interpretability, making it applicable in statistics, machine learning, and seismology.

Moment tensor decomposition denotes a family of methods for representing tensors built from moments, or interpreted as moment objects, in lower-dimensional, lower-rank, or physically structured forms. In statistics and machine learning, the canonical example is the empirical symmetric moment tensor

$$\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d} \approx \mathbb{E}(x^{\otimes d}),$$

with $\mathcal{M} \in S^d(\mathbb{R}^n)$, formed from samples $x_1,\ldots,x_p \in \mathbb{R}^n$ (Jin et al., 2022). Decomposition is then used for economical representation, feature extraction, latent-variable recovery, constrained atomic recovery, or physically interpretable source separation. The literature spans nonconvex low-rank approximation, moment-matrix and sum-of-squares formulations, identifiability theory for symmetric decompositions, and application-specific decompositions in mechanics, seismology, and integral geometry (Jin et al., 2022).

1. Core object and decomposition models

For samples $x_1,\ldots,x_p \in \mathbb{R}^n$, the empirical $d$-th order moment tensor is

$$\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d}.$$

This is an order-$d$ symmetric tensor, and its explicit formation generally incurs $O(n^d)$ storage and, for sample-based construction, $O(pn^d)$ arithmetic cost (Jin et al., 2022). The same structural object appears in related formulations as

$\X = \frac{1}{p} \sum_{\ell=1}^p \Vl{\ell}^{\otimes d},$

with $\mathcal{M} \in S^d(\mathbb{R}^n)$0, emphasizing that it is a sum of symmetric rank-1 outer products and therefore has rank at most $\mathcal{M} \in S^d(\mathbb{R}^n)$1 (Sherman et al., 2019).

One major decomposition model is the low-rank symmetric Tucker approximation. For a prescribed rank $\mathcal{M} \in S^d(\mathbb{R}^n)$2, the problem is written as

$\mathcal{M} \in S^d(\mathbb{R}^n)$3

where $\mathcal{M} \in S^d(\mathbb{R}^n)$4 is orthonormal. The corresponding core tensor is

$\mathcal{M} \in S^d(\mathbb{R}^n)$5

(Jin et al., 2022). This formulation compresses the moment tensor into a lower-dimensional symmetric core while preserving a dominant $\mathcal{M} \in S^d(\mathbb{R}^n)$6-dimensional subspace.

A second major model is symmetric CP decomposition, written as

$\mathcal{M} \in S^d(\mathbb{R}^n)$7

or, for empirical moments,

$\mathcal{M} \in S^d(\mathbb{R}^n)$8

This form is central in Gaussian mixture recovery, latent-variable learning, and algebraic moment methods (Huang et al., 2024, Sherman et al., 2019, Khouja et al., 2021).

A broader algebraic perspective interprets tensor decomposition as a moment problem. In this view, the decomposition vectors are the support of a representing measure, the decomposition length corresponds to the support cardinality, and truncated moment sequences or moment matrices encode feasibility and recoverability (Huang et al., 2024, Bernardi et al., 2011). This unifies tensor decomposition with generalized moment problems, quasi-Hankel matrices, and atomic measures.

2. Symmetric Tucker decomposition of sample moments

The scalable symmetric Tucker program developed for high-order sample moments is motivated by “economical representation” and “feature extraction” from high-order, high-dimensional moment tensors (Jin et al., 2022). The central optimization variable is the orthonormal factor matrix $\mathcal{M} \in S^d(\mathbb{R}^n)$9, and the cost depends only on the subspace spanned by $x_1,\ldots,x_p \in \mathbb{R}^n$0, not on a particular basis. Accordingly, the induced projector sequence $x_1,\ldots,x_p \in \mathbb{R}^n$1 is analyzed on the Grassmannian $x_1,\ldots,x_p \in \mathbb{R}^n$2 (Jin et al., 2022).

The principal numerical schemes are projected gradient descent (PGD) and higher-order eigenvalue decomposition (HOEVD). The PGD step on the Stiefel manifold is

$x_1,\ldots,x_p \in \mathbb{R}^n$3

and, in the formulation specialized to the objective $x_1,\ldots,x_p \in \mathbb{R}^n$4, the update can be written

$x_1,\ldots,x_p \in \mathbb{R}^n$5

(Jin et al., 2022). HOEVD computes $x_1,\ldots,x_p \in \mathbb{R}^n$6 from the leading eigenvectors of a matricized or flattened tensor, but for $x_1,\ldots,x_p \in \mathbb{R}^n$7 it “does not guarantee convergence to critical points of $x_1,\ldots,x_p \in \mathbb{R}^n$8” (Jin et al., 2022).

The Grassmannian formulation yields a cost

$x_1,\ldots,x_p \in \mathbb{R}^n$9

with gradient

$x_1,\ldots,x_p \in \mathbb{R}^n$0

where

$x_1,\ldots,x_p \in \mathbb{R}^n$1

The associated theory establishes first-order criticality for any initialization and sufficiently small fixed step size, and second-order criticality for almost every initialization, excluding a measure-zero set (Jin et al., 2022).

This Tucker line of work differs from symmetric CP methods in emphasizing low-dimensional subspace extraction rather than explicit atomic expansion. A plausible implication is that Tucker decompositions are especially natural when the moment tensor is used as a compressed descriptor, rather than as a direct parameterization of latent atoms.

3. Tensor-free, implicit, and streaming computation

A central obstacle in moment tensor decomposition is that explicit tensor formation becomes prohibitive at large $x_1,\ldots,x_p \in \mathbb{R}^n$2 or $x_1,\ldots,x_p \in \mathbb{R}^n$3. Two closely related works replace tensor construction by direct contractions against raw data. In the symmetric Tucker setting, the “implicit” strategy computes objective values and gradients directly from the observations $x_1,\ldots,x_p \in \mathbb{R}^n$4, and the “streaming” strategy processes the data in mini-batches, discarding each batch after use (Jin et al., 2022). In the symmetric CP setting, the same principle is used to avoid building the empirical moment tensor $x_1,\ldots,x_p \in \mathbb{R}^n$5 and to evaluate all required tensor-times-vector contractions through matrix operations on the sample matrix $x_1,\ldots,x_p \in \mathbb{R}^n$6 (Sherman et al., 2019).

For the streaming batch objective in the Tucker framework,

$x_1,\ldots,x_p \in \mathbb{R}^n$7

and the stochastic gradient is

$x_1,\ldots,x_p \in \mathbb{R}^n$8

where $x_1,\ldots,x_p \in \mathbb{R}^n$9 is the $d$0-th batch (Jin et al., 2022). The paper states that streaming reduces computation and memory from $d$1 to $d$2, where $d$3 is the batch size, with per-iteration memory $d$4 and per-iteration computation $d$5 (Jin et al., 2022).

The symmetric CP analogue replaces explicit TTSV operations by

$d$6

where $d$7 stores the observations as columns and $d$8 contains the sample weights (Sherman et al., 2019). The CP objective is

$d$9

with

$$\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d}.$$0

The stated computational gain is a reduction from explicit $$\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d}.$$1 time and $$\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d}.$$2 memory per iteration to implicit $$\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d}.$$3 time and $$\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d}.$$4 memory (Sherman et al., 2019).

The nonparametric mixture-model literature generalizes this tensor-free principle to incomplete or masked moment equations. There, the decomposition is performed not on the full tensor but on off-diagonal or partition-projected moment tensors, and alternating least squares is implemented by small linear systems whose normal equations are computed through kernel-style identities rather than explicit $$\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d}.$$5 arrays (Zhang et al., 2022). The stated per-iteration costs are $$\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d}.$$6 in time and $$\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d}.$$7 in storage (Zhang et al., 2022).

4. Moment matrices, SOS relaxations, and algebraic recovery

A distinct but closely connected tradition reformulates tensor decomposition as a truncated moment or moment-matrix problem. In the multihomogeneous and partially symmetric setting, a tensor $$\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d}.$$8 is identified with a linear functional $$\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d}.$$9, and decomposition becomes the problem of extending the data to a linear form $d$0 such that the associated Hankel or quasi-Hankel matrix

$d$1

has minimal rank and a kernel corresponding to the ideal of the support points $d$2 (Bernardi et al., 2011). The rank of $d$3 equals the number of rank-1 terms in the minimal decomposition, while its kernel gives the vanishing polynomials of the decomposition support (Bernardi et al., 2011).

The moment-matrix extension framework uses flat extension and commutation relations of multiplication operators. In the algorithm described for general multihomogeneous tensors, one builds monomial sets $d$4, forms a partial moment matrix $d$5, checks flat extension or commuting multiplication operators, computes the kernel, and extracts the support points from common eigenvectors of multiplication matrices (Bernardi et al., 2011). This directly connects decomposition to border bases and quotient-algebra methods.

Moment-SOS relaxations generalize this picture to decompositions constrained by semialgebraic sets $d$6. The tensor recovery problem seeks

$d$7

subject to linear constraints $d$8, while the moment recovery problem seeks a positive Borel measure $d$9 supported on

$O(n^d)$0

such that $O(n^d)$1 (Huang et al., 2024). The optimization is posed as

$O(n^d)$2

with $O(n^d)$3 a generic sum-of-squares polynomial (Huang et al., 2024).

The corresponding Lasserre hierarchy replaces measures by truncated moment sequences and imposes moment-matrix and localizing-matrix constraints. Under compactness, archimedean hypotheses, and a generic objective $O(n^d)$4, flat extension yields atomic representing measures, hence tensor decompositions. The paper states that, by Carathéodory’s theorem, the number of atoms in the moment setup is at most $O(n^d)$5, the number of constraints, and that for generic SOS $O(n^d)$6, the solution is unique and atomic (Huang et al., 2024).

More recent work extends moment-based decomposition to specialized cones and sparsity structures. For the pseudo-moment cone, generic moment matrices formed by $O(n^d)$7 weighted atoms have a simplicial minimal face generated by the planted rank-one atoms, enabling a Carathéodory-type extreme-ray decomposition algorithm (Kang et al., 7 May 2026). For completely positive tensors, an ideal-sparse generalized moment formulation exploits maximal cliques of a support multi-hypergraph and solves the problem by lower-dimensional clique-wise moment relaxations (Huang et al., 21 May 2025). In both cases, the decomposition is mediated by facial geometry and moment representability rather than by direct multilinear optimization.

5. Identifiability and latent-variable learning

Moment tensor decomposition is central in recovering latent-variable models from low-order statistics. For spherical Gaussian mixtures, the third-order moment polynomial

$O(n^d)$8

is a sum of rank-1 symmetric tensors, and the cited work shows that moment tensors built from spherical Gaussian mixtures are identifiable when the interpolation degree is strictly less than half the tensor order (Khouja et al., 2021). The relevant rank condition is formulated through Hankel matrices $O(n^d)$9, and the decomposition algorithm proceeds by SVD, rank detection, simultaneous diagonalization, and recovery of the weights $O(pn^d)$0 through a linear system (Khouja et al., 2021).

A different route uses pairs of moment-like tensors sharing the same rank-1 vectors but different coefficients. For

$O(pn^d)$1

flattening yields

$O(pn^d)$2

so that

$O(pn^d)$3

The eigenvectors recover the shared components, and this joint-diagonalization principle underlies robust tensor decomposition for underdetermined ICA and spherical Gaussian mixtures (Goyal et al., 2013).

When data are partially observed, weighted moments compensate for heterogeneous estimation quality. The weighted tensor power method introduces

$O(pn^d)$4

with factorizable weights

$O(pn^d)$5

so that standard tensor decomposition can be applied after dimension-wise rescaling (Gottesman et al., 2017). Under the paper’s assumptions, the optimal weights minimizing an upper bound on reconstruction error satisfy

$O(pn^d)$6

where $O(pn^d)$7 is the probability that dimension $O(pn^d)$8 is observed (Gottesman et al., 2017).

For conditionally independent nonparametric mixtures, the tensor to be decomposed is incomplete by design: only off-diagonal or masked components are matched. The paper formulates identifiability of the off-diagonal projection

$O(pn^d)$9

and proves local linear convergence of an ALS scheme under the corresponding conditions (Zhang et al., 2022). This suggests that moment tensor decomposition remains viable even when the full tensor is neither formed nor fully observed.

6. Related decomposition meanings in physics, mechanics, and tensor field theory

The phrase “moment tensor decomposition” also appears outside statistical moment estimation. In mining-induced seismicity, the moment tensor is a physical source tensor rather than an empirical statistical moment. The cited decomposition writes

$\X = \frac{1}{p} \sum_{\ell=1}^p \Vl{\ell}^{\otimes d},$0

where $\X = \frac{1}{p} \sum_{\ell=1}^p \Vl{\ell}^{\otimes d},$1 is a double-couple component and $\X = \frac{1}{p} \sum_{\ell=1}^p \Vl{\ell}^{\otimes d},$2 is a closing-crack component (Rigby, 2023). Not every moment tensor admits such a decomposition; the admissible set is characterized on the Hudson plot by the boundary equations

$\X = \frac{1}{p} \sum_{\ell=1}^p \Vl{\ell}^{\otimes d},$3

$\X = \frac{1}{p} \sum_{\ell=1}^p \Vl{\ell}^{\otimes d},$4

$\X = \frac{1}{p} \sum_{\ell=1}^p \Vl{\ell}^{\otimes d},$5

and, within this admissible region, the family of all decompositions is defined implicitly by

$\X = \frac{1}{p} \sum_{\ell=1}^p \Vl{\ell}^{\otimes d},$6

(Rigby, 2023). Here decomposition is valued for physical interpretability rather than for low-rank compression.

In continuum mechanics, tensor decomposition is used to split stress and strain into deviatoric and volumetric parts through multiplying decomposition tensors

$\X = \frac{1}{p} \sum_{\ell=1}^p \Vl{\ell}^{\otimes d},$7

yielding

$\X = \frac{1}{p} \sum_{\ell=1}^p \Vl{\ell}^{\otimes d},$8

(Lee et al., 2012). Although this is not a decomposition of statistical moment tensors, it is another established tensor decomposition paradigm attached to the term “moment tensor” in mechanics-adjacent usage.

In integral geometry, symmetric tensor fields admit a generalized $\X = \frac{1}{p} \sum_{\ell=1}^p \Vl{\ell}^{\otimes d},$9-solenoidal/$\mathcal{M} \in S^d(\mathbb{R}^n)$00-potential decomposition

$\mathcal{M} \in S^d(\mathbb{R}^n)$01

which is then used to characterize the kernel of integral moment transforms (Mishra et al., 2020). In angular-momentum theory, tensor products of a vector are decomposed into irreducible angular components

$\mathcal{M} \in S^d(\mathbb{R}^n)$02

with each component carrying definite angular momentum $\mathcal{M} \in S^d(\mathbb{R}^n)$03 (Adkins, 2016). These are not sample-moment decompositions in the statistical sense, but they show that decomposition of symmetric tensorial moment objects is a cross-disciplinary theme.

Taken together, these strands show that moment tensor decomposition is not a single algorithmic object but a technical umbrella. In statistical learning it most often denotes low-rank decomposition of empirical symmetric moments; in algebraic optimization it denotes atomic recovery through moment matrices and SOS relaxations; and in physical sciences it denotes decomposition into invariant, constrained, or interpretable tensorial components (Jin et al., 2022, Huang et al., 2024, Rigby, 2023).