---
title: Multi-Subspace Power Method (MSPM)
url: https://www.emergentmind.com/topics/multi-subspace-power-method-mspm
type: topic
---

# Multi-Subspace Power Method (MSPM)

The Multi-Subspace Power Method (MSPM) is a tensor decomposition algorithm for recovering low-rank, partially symmetric CP decompositions from real tensors exhibiting arbitrary partial symmetry—ranging from fully symmetric to fully asymmetric cases. MSPM generalizes the subspace power method beyond symmetric tensors by introducing an orthonormal-slice transformation and a globally convergent, shifted higher-order power iteration for extracting partially symmetric singular-vector tuples (pSVTs). The method facilitates efficient and accurate recovery of tensor factors and demonstrates theoretical and empirical advantages over classical approaches[2510.18627].

## 1. Problem Formulation and Preliminaries

Consider positive integers $d_1,\dots,d_\ell$, $m_1,\dots,m_\ell$, and $r$. Let the input tensor be
\[
\mathcal{T} \in S^{d_1}(\mathbb{R}^{m_1}) \otimes \cdots \otimes S^{d_\ell}(\mathbb{R}^{m_\ell}),
\]
where $S^{d_i}(\mathbb{R}^{m_i})$ denotes the space of order-$d_i$ symmetric tensors over $\mathbb{R}^{m_i}$. The target is the partially symmetric CP decomposition:
\begin{equation}
\mathcal{T} = \sum_{i=1}^{r} \lambda_i\, (v_i^{(1)})^{\otimes d_1} \otimes \cdots \otimes (v_i^{(\ell)})^{\otimes d_\ell},
\end{equation}
with vectors $v_i^{(j)} \in \mathbb{S}^{m_j-1}$ and $\lambda_i \in \mathbb{R}\setminus\{0\}$. The decomposition problem is generally NP-hard, but for low $r$ and generic tensors, algorithmic solutions exist that can guarantee recovery[2510.18627].

## 2. Orthonormal-Slice Transformation

MSPM starts by choosing a flattening pattern $f=(f_1,\ldots,f_\ell)$, $0\leq f_i\leq d_i$, and computes the $f$-flattening of $\mathcal{T}$:
\[
\mathbf{T}^{(f)} = \mathrm{flatten}_f(\mathcal{T}) \in \mathbb{R}^{N_\mathrm{row} \times N_\mathrm{col}}
\]
with
\[
N_\mathrm{row} = \prod_{i=1}^\ell \binom{m_i + f_i - 1}{f_i}, \qquad N_\mathrm{col} = \prod_{i=1}^\ell \binom{m_i + (d_i - f_i) - 1}{d_i-f_i}.
\]
A thin singular value decomposition (SVD) yields leading $r$ left singular vectors, which are reshaped into a tensor $\mathcal{T}^{(f)}$ of dimension 
\[
\bigotimes_{i=1}^\ell S^{f_i}(\mathbb{R}^{m_i}) \otimes \mathbb{R}^r
\]
with orthonormal slices along the last ($r$th) mode. The original rank-one tensor summands map to partially symmetric rank-one components in $\mathcal{T}^{(f)}$ with shared factors $v^{(i)}$ and a standard basis vector in $\mathbb{R}^r$[2510.18627].

## 3. Partially Symmetric Singular-Vector Tuples

A partially symmetric singular-vector tuple (pSVT) is a tuple $(v^{(1)},\dots,v^{(\ell)}) \in \mathbb{S}^{m_1-1}\times\cdots\times \mathbb{S}^{m_\ell-1}$ and scalar $\sigma$ solving, for each $j$,
\begin{equation}
\mathcal{A} \cdot (v^{(1)})^{\otimes(a_1-\delta_{1j})} \otimes\cdots\otimes (v^{(\ell)})^{\otimes(a_\ell-\delta_{\ell j})} = \sigma\, v^{(j)},
\end{equation}
where $\mathcal{A}$ is a partially symmetric tensor and $\delta_{ij}$ is the Kronecker delta. For the orthonormal-slice tensor $\mathcal{T}^{(f)}$, all pSVTs satisfy $|\sigma| \leq 1$; those with $\sigma=1$ correspond exactly to the rank-one components of the decomposition[2510.18627].

## 4. The Multi-Subspace Power Method: Algorithmic Structure

The MSPM algorithm proceeds as follows:

1. **Subspace Extraction:** Compute the $f$-flattening $\mathbf{T}^{(f)}$ of $\mathcal{T}$, perform SVD and truncate to top $r$ singular vectors; construct $\mathcal{T}^{(f)}$ with orthonormal slices.
2. **Rank-one Recovery:** For $i=1$ to $r$, apply the shifted higher-order power method (PS-HOPM) to $\mathcal{T}^{(f)}$ to find a pSVT $(v_i^{(1)},\ldots,v_i^{(\ell)};\sigma=1)$.
3. **Completion:** Extend recovered vectors to the full tensor rank-one term.
4. **Coefficient Recovery:** Calculate the $\lambda_i$ via projection.
5. **Deflation and Update:** Remove the recovered component via a one-rank SVD downdate.
6. **Termination:** Repeat until $r$ components are recovered.

The primary subroutine, PS-HOPM, performs alternating block updates with shift parameters $\gamma_i$ to ensure convexity, followed by normalization, and updates the $\sigma$ value as a fully contracted tensor product[2510.18627].

## 5. Convergence Properties

- **Global Convergence:** Given shift parameters $\gamma_i \geq (a_i-1)\|\mathcal{A}\|_2$, the shifted objective,
  \[
  F(v^{(1)},\dots,v^{(\ell)}) = \langle \mathcal{A}, (v^{(1)})^{\otimes a_1}\otimes\cdots\otimes(v^{(\ell)})^{\otimes a_\ell}\rangle + \sum_{i=1}^\ell \gamma_i \|v^{(i)}\|^{a_i},
  \]
  is maximized by alternating block ascent, converging to pSVTs from any initialization[2510.18627].
- **Local Linear Convergence:** In the special case $\ell=2$, every strict local maximizer is a fixed point for PS-HOPM under which convergence is locally linear. This is established via spectral analysis of the Jacobian at the fixed point.

## 6. Computational Complexity and Empirical Results

Let $M := \prod_{i=1}^\ell m_i^{d_i}$ be the total number of tensor entries.

- **Orthonormal-Slice Construction:** The thin SVD step has $O(\min(N_\mathrm{row},N_\mathrm{col})^2\max(N_\mathrm{row},N_\mathrm{col}))$ cost.
- **PS-HOPM Iteration:** Each update costs $O(M)$ arithmetic, relying on two high-dimensional contractions.
- **Deflation:** SVD downdate per step requires $O(r^2)$.
- **Total Cost:** $O(\mathrm{SVD\ cost} + r\cdot M)$.

Empirically, for tensors up to size $100^3$ and rank $80$, MSPM achieves higher accuracy and runtime performance than Alternating Least Squares (ALS), nonlinear least-squares solvers (TensorLab NLS/MINF), simultaneous diagonalization (FFDIAG, Jacobi, Jennrich), and generalized SVD (HOGSVD). In log-error versus log-time plots, MSPM occupies the best-accuracy, best-time region[2510.18627].

## 7. Limitations and Directions for Generalization

- **Rank Constraints:** The theoretical recovery guarantee holds when $r \leq r_{\max}(f)$; above this, uniqueness is lost and performance may degrade.
- **Exploitation of Symmetry:** MSPM leverages tensor partial symmetry via flattenings tailored to the symmetry type. In the fully asymmetric case ($f=(1,\ldots,1)$), it reduces to a deflationary CP power method, still performing competitively in experiments.
- **Unresolved Issues:** Open questions include extending local linear convergence to $\ell>2$ blocks, more precise robustness/nonideal SVD truncation analysis, and adaptive shift strategies for accelerated PS-HOPM convergence without tight spectral norm estimation.

A plausible implication is that MSPM unifies and subsumes a range of tensor decomposition strategies under a common algebraic-geometric framework, facilitating future advances in the study of low-rank tensor varieties and algorithms[2510.18627].

Source: https://www.emergentmind.com/topics/multi-subspace-power-method-mspm