---
title: Moment Tensor Decomposition Overview
url: https://www.emergentmind.com/topics/moment-tensor-decomposition
type: topic
---

# Moment Tensor Decomposition Overview

Moment tensor decomposition denotes a family of methods for representing tensors built from moments, or interpreted as moment objects, in lower-dimensional, lower-rank, or physically structured forms. In statistics and machine learning, the canonical example is the empirical symmetric moment tensor
\[
\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d} \approx \mathbb{E}(x^{\otimes d}),
\]
with \(\mathcal{M} \in S^d(\mathbb{R}^n)\), formed from samples \(x_1,\ldots,x_p \in \mathbb{R}^n\) [2204.10824]. Decomposition is then used for economical representation, feature extraction, latent-variable recovery, constrained atomic recovery, or physically interpretable source separation. The literature spans nonconvex low-rank approximation, moment-matrix and sum-of-squares formulations, identifiability theory for symmetric decompositions, and application-specific decompositions in mechanics, seismology, and integral geometry [2204.10824].

## 1. Core object and decomposition models

For samples \(x_1,\ldots,x_p \in \mathbb{R}^n\), the empirical \(d\)-th order moment tensor is
\[
\mathcal{M} = \frac{1}{p}\sum_{i=1}^p x_i^{\otimes d}.
\]
This is an order-\(d\) symmetric tensor, and its explicit formation generally incurs \(O(n^d)\) storage and, for sample-based construction, \(O(pn^d)\) arithmetic cost [2204.10824]. The same structural object appears in related formulations as
\[
\X = \frac{1}{p} \sum_{\ell=1}^p \Vl{\ell}^{\otimes d},
\]
with \(\X \in \mathbb{S}^{[d,n]}\), emphasizing that it is a sum of symmetric rank-1 outer products and therefore has rank at most \(p\) [1911.03813].

One major decomposition model is the low-rank symmetric Tucker approximation. For a prescribed rank \(r\), the problem is written as
\[
\max_{Q^\top Q = I_r} F(Q) \equiv \Big\| \mathcal{M} \cdot (Q, \ldots, Q) \Big\|^2,
\]
where \(Q \in \mathbb{R}^{n \times r}\) is orthonormal. The corresponding core tensor is
\[
\mathcal{C} = \mathcal{M} \cdot (Q^\top, \ldots, Q^\top) \in S^d(\mathbb{R}^r)
\]
[2204.10824]. This formulation compresses the moment tensor into a lower-dimensional symmetric core while preserving a dominant \(r\)-dimensional subspace.

A second major model is symmetric CP decomposition, written as
\[
\mathcal{A} = \sum_{i=1}^r \lambda_i u_i^{\otimes d},
\]
or, for empirical moments,
\[
\X \approx \M = \sum_{j=1}^r \lambda_j \Aj^{\otimes d}.
\]
This form is central in Gaussian mixture recovery, latent-variable learning, and algebraic moment methods [2404.18332; 1911.03813; 2106.00555].

A broader algebraic perspective interprets tensor decomposition as a moment problem. In this view, the decomposition vectors are the support of a representing measure, the decomposition length corresponds to the support cardinality, and truncated moment sequences or moment matrices encode feasibility and recoverability [2404.18332; 1105.1229]. This unifies tensor decomposition with generalized moment problems, quasi-Hankel matrices, and atomic measures.

## 2. Symmetric Tucker decomposition of sample moments

The scalable symmetric Tucker program developed for high-order sample moments is motivated by “economical representation” and “feature extraction” from high-order, high-dimensional moment tensors [2204.10824]. The central optimization variable is the orthonormal factor matrix \(Q\), and the cost depends only on the subspace spanned by \(Q\), not on a particular basis. Accordingly, the induced projector sequence \(P_t = Q_t Q_t^\top\) is analyzed on the Grassmannian \(Gr(n,r)\) [2204.10824].

The principal numerical schemes are projected gradient descent (PGD) and higher-order eigenvalue decomposition (HOEVD). The PGD step on the Stiefel manifold is
\[
Q_{t} \gets \operatorname{qr}\left(Q_{t-1} + \gamma_t \nabla F(Q_{t-1}) \right),
\]
and, in the formulation specialized to the objective \(F(Q)\), the update can be written
\[
Q_{t} \gets \operatorname{qr} \left(Q_{t-1} + 2d \gamma_t \langle \mathcal{M}(I_n, Q_{t-1}, \ldots, Q_{t-1}), \mathcal{M}(Q_{t-1}, \ldots, Q_{t-1}) \rangle_{-1} \right)
\]
[2204.10824]. HOEVD computes \(Q\) from the leading eigenvectors of a matricized or flattened tensor, but for \(d>2\) it “does not guarantee convergence to critical points of \(F(Q)\)” [2204.10824].

The Grassmannian formulation yields a cost
\[
f(P) = \| \mathcal{M} \cdot (P, ..., P) \|^2,
\]
with gradient
\[
\text{grad } f(P) = \text{sym}\left( 2 (I_n - P) w(P)P \right),
\]
where
\[
w(P) = d \langle \mathcal{M}(I_n, P, ..., P), \mathcal{M}(I_n, P, ..., P) \rangle_{-1}.
\]
The associated theory establishes first-order criticality for any initialization and sufficiently small fixed step size, and second-order criticality for almost every initialization, excluding a measure-zero set [2204.10824].

This Tucker line of work differs from symmetric CP methods in emphasizing low-dimensional subspace extraction rather than explicit atomic expansion. A plausible implication is that Tucker decompositions are especially natural when the moment tensor is used as a compressed descriptor, rather than as a direct parameterization of latent atoms.

## 3. Tensor-free, implicit, and streaming computation

A central obstacle in moment tensor decomposition is that explicit tensor formation becomes prohibitive at large \(n\) or \(d\). Two closely related works replace tensor construction by direct contractions against raw data. In the symmetric Tucker setting, the “implicit” strategy computes objective values and gradients directly from the observations \(x_i\), and the “streaming” strategy processes the data in mini-batches, discarding each batch after use [2204.10824]. In the symmetric CP setting, the same principle is used to avoid building the empirical moment tensor \(\X\) and to evaluate all required tensor-times-vector contractions through matrix operations on the sample matrix \(V\) [1911.03813].

For the streaming batch objective in the Tucker framework,
\[
F_t(Q) = \left\| \frac{1}{b} \sum_{i=1}^b (Q^\top x_{t,i})^{\otimes d} \right\|^2,
\]
and the stochastic gradient is
\[
\nabla F_t(Q) = \frac{2d}{b^2} X_t \left(X_t^\top Q Q^\top X_t\right)^{[d-1]} X_t^\top Q,
\]
where \(X_t \in \mathbb{R}^{n \times b}\) is the \(t\)-th batch [2204.10824]. The paper states that streaming reduces computation and memory from \(O(p n^d)\) to \(O(nb + nr)\), where \(b\) is the batch size, with per-iteration memory \(O(nb + nr)\) and per-iteration computation \(O(rnb + rb^2)\) [2204.10824].

The symmetric CP analogue replaces explicit TTSV operations by
\[
\X a^{d-1} = \V \diag(\nu) [\V^T a]^{d-1},
\]
where \(\V\) stores the observations as columns and \(\nu\) contains the sample weights [1911.03813]. The CP objective is
\[
\min_{\lambda, A} \| \X - \M \|^2,
\]
with
\[
\|\M\|^2 = \lambda^T [A^T A]^d \lambda,
\qquad
\|\X - \M\|^2 = \|\X\|^2 + \|\M\|^2 - 2\lambda^T w.
\]
The stated computational gain is a reduction from explicit \(O(n^d r)\) time and \(O(n^d)\) memory per iteration to implicit \(O(npr)\) time and \(O(np)\) memory [1911.03813].

The nonparametric mixture-model literature generalizes this tensor-free principle to incomplete or masked moment equations. There, the decomposition is performed not on the full tensor but on off-diagonal or partition-projected moment tensors, and alternating least squares is implemented by small linear systems whose normal equations are computed through kernel-style identities rather than explicit \(n^d\) arrays [2210.14386]. The stated per-iteration costs are \(\mathcal{O}(npr + nr^3)\) in time and \(\mathcal{O}(n(r + p))\) in storage [2210.14386].

## 4. Moment matrices, SOS relaxations, and algebraic recovery

A distinct but closely connected tradition reformulates tensor decomposition as a truncated moment or moment-matrix problem. In the multihomogeneous and partially symmetric setting, a tensor \(T\) is identified with a linear functional \(T^*\), and decomposition becomes the problem of extending the data to a linear form \(\Lambda\) such that the associated Hankel or quasi-Hankel matrix
\[
(H_\Lambda)_{\alpha,\beta} = \Lambda(x^\alpha x^\beta)
\]
has minimal rank and a kernel corresponding to the ideal of the support points \(\zeta_i\) [1105.1229]. The rank of \(H_\Lambda\) equals the number of rank-1 terms in the minimal decomposition, while its kernel gives the vanishing polynomials of the decomposition support [1105.1229].

The moment-matrix extension framework uses flat extension and commutation relations of multiplication operators. In the algorithm described for general multihomogeneous tensors, one builds monomial sets \(B,B'\), forms a partial moment matrix \(\mathbb{H}^{B',B}\), checks flat extension or commuting multiplication operators, computes the kernel, and extracts the support points from common eigenvectors of multiplication matrices [1105.1229]. This directly connects decomposition to border bases and quotient-algebra methods.

Moment-SOS relaxations generalize this picture to decompositions constrained by semialgebraic sets \(K\). The tensor recovery problem seeks
\[
\mathcal{A} = \sum_{i=1}^r \lambda_i u_i^{\otimes d}, \quad u_i \in K, \lambda_i > 0,
\]
subject to linear constraints \(\langle F_i,\mathcal{A}\rangle = b_i\), while the moment recovery problem seeks a positive Borel measure \(\mu\) supported on
\[
K = \{ x \in \mathbb{R}^n : c_i(x) = 0, \; i \in E; \; c_j(x) \geq 0, \; j \in I \}
\]
such that \(\int a_i(x)\,d\mu = b_i\) [2404.18332]. The optimization is posed as
\[
\min_{\mu \in \mathcal{B}(K)} \int R(x) d\mu
\quad \text{subject to} \quad
\int a_i(x) d\mu = b_i,
\]
with \(R(x)\) a generic sum-of-squares polynomial [2404.18332].

The corresponding Lasserre hierarchy replaces measures by truncated moment sequences and imposes moment-matrix and localizing-matrix constraints. Under compactness, archimedean hypotheses, and a generic objective \(R\), flat extension yields atomic representing measures, hence tensor decompositions. The paper states that, by Carathéodory’s theorem, the number of atoms in the moment setup is at most \(m\), the number of constraints, and that for generic SOS \(R\), the solution is unique and atomic [2404.18332].

More recent work extends moment-based decomposition to specialized cones and sparsity structures. For the pseudo-moment cone, generic moment matrices formed by \(O(n^d)\) weighted atoms have a simplicial minimal face generated by the planted rank-one atoms, enabling a Carathéodory-type extreme-ray decomposition algorithm [2605.06854]. For completely positive tensors, an ideal-sparse generalized moment formulation exploits maximal cliques of a support multi-hypergraph and solves the problem by lower-dimensional clique-wise moment relaxations [2505.15056]. In both cases, the decomposition is mediated by facial geometry and moment representability rather than by direct multilinear optimization.

## 5. Identifiability and latent-variable learning

Moment tensor decomposition is central in recovering latent-variable models from low-order statistics. For spherical Gaussian mixtures, the third-order moment polynomial
\[
M_3(\mathbf{X}) = \sum_{i=1}^r \omega_i (\mu_i \cdot \mathbf{X})^3
\]
is a sum of rank-1 symmetric tensors, and the cited work shows that moment tensors built from spherical Gaussian mixtures are identifiable when the interpolation degree is strictly less than half the tensor order [2106.00555]. The relevant rank condition is formulated through Hankel matrices \(H_T^{k,d-k}\), and the decomposition algorithm proceeds by SVD, rank detection, simultaneous diagonalization, and recovery of the weights \(\omega_i\) through a linear system [2106.00555].

A different route uses pairs of moment-like tensors sharing the same rank-1 vectors but different coefficients. For
\[
T_\mu = \sum_{i=1}^m \mu_i A_i^{\otimes d},
\qquad
T_\lambda = \sum_{i=1}^m \lambda_i A_i^{\otimes d},
\]
flattening yields
\[
M_\mu = A^{\odot d/2} \cdot \text{diag}(\mu_i) \cdot (A^{\odot d/2})^\top,
\qquad
M_\lambda = A^{\odot d/2} \cdot \text{diag}(\lambda_i) \cdot (A^{\odot d/2})^\top,
\]
so that
\[
M_\mu M_\lambda^{-1}
=
A^{\odot d/2}
\cdot
\text{diag}\left(\frac{\mu_i}{\lambda_i}\right)
\cdot
(A^{\odot d/2})^{-1}.
\]
The eigenvectors recover the shared components, and this joint-diagonalization principle underlies robust tensor decomposition for underdetermined ICA and spherical Gaussian mixtures [1306.5825].

When data are partially observed, weighted moments compensate for heterogeneous estimation quality. The weighted tensor power method introduces
\[
\hat{S}^{*}_{d_1d_2} = w_{d_1d_2} \hat{S}_{d_1d_2},
\qquad
\hat{T}^{*}_{d_1d_2d_3} = w_{d_1d_2d_3} \hat{T}_{d_1d_2d_3},
\]
with factorizable weights
\[
w_{d_1d_2} = w_{d_1} w_{d_2},
\qquad
w_{d_1d_2d_3} = w_{d_1} w_{d_2} w_{d_3},
\]
so that standard tensor decomposition can be applied after dimension-wise rescaling [1710.06818]. Under the paper’s assumptions, the optimal weights minimizing an upper bound on reconstruction error satisfy
\[
w_d \propto p_d,
\]
where \(p_d\) is the probability that dimension \(d\) is observed [1710.06818].

For conditionally independent nonparametric mixtures, the tensor to be decomposed is incomplete by design: only off-diagonal or masked components are matched. The paper formulates identifiability of the off-diagonal projection
\[
P \left(\sum_{j=1}^r a_j^{\otimes d}\right)
\]
and proves local linear convergence of an ALS scheme under the corresponding conditions [2210.14386]. This suggests that moment tensor decomposition remains viable even when the full tensor is neither formed nor fully observed.

## 6. Related decomposition meanings in physics, mechanics, and tensor field theory

The phrase “moment tensor decomposition” also appears outside statistical moment estimation. In mining-induced seismicity, the moment tensor is a physical source tensor rather than an empirical statistical moment. The cited decomposition writes
\[
\mathbf{M} = \mathbf{M}^{K^{-}} + \mathbf{M}^{D},
\]
where \(\mathbf{M}^{D}\) is a double-couple component and \(\mathbf{M}^{K^{-}}\) is a closing-crack component [2302.09417]. Not every moment tensor admits such a decomposition; the admissible set is characterized on the Hudson plot by the boundary equations
\[
h_{1D}(u,v) = v - \frac{\nu+1}{4(1-2\nu)}u = 0,
\]
\[
h_{DK}(u,v) = v + \frac{\nu+1}{2(1-2\nu)}u = 0,
\]
\[
h_{2^{-}K^{-}}(u,v) = v - \frac{\nu+1}{2(2-\nu)}u + \frac{\nu+1}{2-\nu} = 0,
\]
and, within this admissible region, the family of all decompositions is defined implicitly by
\[
d(\phi, \beta) = \det\left[\mathbf{M} - \mathbf{M}^{K^{-}\left(-\frac{\mathrm{tr}(\mathbf{M})}{(\nu+1)\alpha_{\nu}}, \phi, \beta\right)}\right] = 0
\]
[2302.09417]. Here decomposition is valued for physical interpretability rather than for low-rank compression.

In continuum mechanics, tensor decomposition is used to split stress and strain into deviatoric and volumetric parts through multiplying decomposition tensors
\[
M_{ijkl}^{d} = \delta_{ki}\delta_{lj} - \frac{1}{3}\delta_{lk}\delta_{ij},
\qquad
M_{ijkl}^{v} = \frac{1}{3}\delta_{lk}\delta_{ij},
\]
yielding
\[
s_{ij} = M_{ijkl}^d\,\sigma_{kl},
\qquad
p_{ij} = M_{ijkl}^v\,\sigma_{kl}
\]
[1211.2693]. Although this is not a decomposition of statistical moment tensors, it is another established tensor decomposition paradigm attached to the term “moment tensor” in mechanics-adjacent usage.

In integral geometry, symmetric tensor fields admit a generalized \(k\)-solenoidal/\(k\)-potential decomposition
\[
f = g + d^{k}v,
\qquad
\delta^{k}g = 0,
\]
which is then used to characterize the kernel of integral moment transforms [2006.13102]. In angular-momentum theory, tensor products of a vector are decomposed into irreducible angular components
\[
p_{i_1} \ldots p_{i_L} = \sum_{\ell=L,L-2,\ldots} \big( p_{i_1} \ldots p_{i_L} \big)_\ell,
\]
with each component carrying definite angular momentum \(\ell\) [1608.07333]. These are not sample-moment decompositions in the statistical sense, but they show that decomposition of symmetric tensorial moment objects is a cross-disciplinary theme.

Taken together, these strands show that moment tensor decomposition is not a single algorithmic object but a technical umbrella. In statistical learning it most often denotes low-rank decomposition of empirical symmetric moments; in algebraic optimization it denotes atomic recovery through moment matrices and SOS relaxations; and in physical sciences it denotes decomposition into invariant, constrained, or interpretable tensorial components [2204.10824; 2404.18332; 2302.09417].

Source: https://www.emergentmind.com/topics/moment-tensor-decomposition