---
title: Asymmetric Low-rank Tensor Factorization (ALTF)
url: https://www.emergentmind.com/topics/asymmetric-low-rank-tensor-factorization-altf
type: topic
---

# Asymmetric Low-rank Tensor Factorization (ALTF)

Searching arXiv for the cited papers and related ALTF usage.

arxiv_search query: "1412.7270 Nearly Low Rank Tensors and Their Approximations"

Asymmetric Low-rank Tensor Factorization (ALTF) denotes, in the superpixel-informed continuous low-rank tensor representation framework, a low-rank tensor model for incomplete multi-dimensional data in which each superpixel receives its own Tucker core and factor matrices, while the factor matrices are generated through a shared neural backbone together with patch-specific heads. In this formulation, ALTF is explicitly asymmetric because the factor-generation path is split between one global module and multiple local modules. Related earlier work studies nonsymmetric or asymmetric low-rank tensor approximation through CP-style decompositions, polynomial relations, Schur decompositions, and random-projection-based simultaneous matrix diagonalization, placing ALTF within the broader landscape of low-rank tensor approximation problems and tensor factorization algorithms [2508.12261].

## 1. Formal definition and tensor model

The 2025 ALTF formulation considers an incomplete 3-D tensor \(\mathcal{T}\in\mathbb{R}^{I\times J\times K}\) together with a partition of its spatial indices into \(K\) superpixels, each patch indexed by \(k=1,\dots,K\). For each superpixel \(k\), the observed entries are denoted by \(\Omega_k\subset\{1,\dots,I\}\times\{1,\dots,J\}\times\{1,\dots,K\}\). As a point of reference, classical Tucker decomposition is written as
\[
\mathcal{X}\approx\mathcal{C}\times_{1}U\times_{2}V\times_{3}W
\]
with \(U\in\mathbb R^{I\times r_1}\), \(V\in\mathbb R^{J\times r_2}\), and \(W\in\mathbb R^{K\times r_3}\).

ALTF breaks the symmetry of “one global core + three global factors” by assigning each superpixel \(k\) its own small core \(\mathcal{C}_k\) and factor matrices \(U_k\), \(V_k\), \(W_k\), while amortizing their generation via a shared backbone network plus patch-specific heads. The coordinate arrays for patch \(k\) are
\[
C^u_k=[u_1,\dots,u_{H_k}]^\top,\quad
C^v_k=[v_1,\dots,v_{W_k}]^\top,\quad
C^w_k=[w_1,\dots,w_{L_k}]^\top,
\]
where \(H_k\times W_k\times L_k\) is the spatial size of superpixel \(k\).

The reconstructed patch is a Tucker composition,
\[
\hat{\mathcal{X}}_k=\mathcal{C}_k\times_{1}U_k\times_{2}V_k\times_{3}W_k\in\mathbb R^{H_k\times W_k\times L_k}.
\]
The defining asymmetry is described as a split between a single global MLP \(\Phi\), which captures shared spectral/spatial coordinate priors, and multiple local heads \(\{\psi^U_k,\psi^V_k,\psi^W_k\}\), each adapting to one superpixel [2508.12261].

## 2. Neural parameterization of the factor matrices

The shared backbone is an MLP
\[
\Phi:\mathbb R^{N\times 1}\to\mathbb R^{N\times D},\quad D=256,
\]
which maps any 1-D coordinate array to a \(D\)-dimensional embedding per coordinate. Figure 3 is summarized as follows: the backbone takes as input a vector of \(N\) sorted 1-D coordinates; it contains a SineLayer, defined as a linear layer \(x\mapsto Wx+b\) followed by \(\sin(\omega_0\cdot)\); four ResidualSineBlocks, each with two linear layers, sine activations, and a skip-connection; then a ReLU; and finally an AttentionBlock projecting to a \(D\)-dimensional per-coordinate feature [2508.12261].

For each patch \(k\), there are three lightweight head-networks,
\[
\psi^U_k,\psi^V_k,\psi^W_k,
\]
which transform the shared embedding into the factor matrices
\[
U_k=\psi^U_k(\Phi(C^u_k))\in\mathbb R^{H_k\times r^k_1},\quad
V_k=\psi^V_k(\Phi(C^v_k))\in\mathbb R^{W_k\times r^k_2},\quad
W_k=\psi^W_k(\Phi(C^w_k))\in\mathbb R^{L_k\times r^k_3}.
\]
Each core \(\mathcal C_k\in\mathbb R^{r^k_1\times r^k_2\times r^k_3}\) is a small, learnable tensor.

The head \(\psi^U_k\) is a single linear layer per superpixel mapping \(\mathbb R^D\to\mathbb R^{r^k_1}\), so each row of \(U_k\) is
\[
[U_k]_{i,:}=W^U_k\,\Phi(C^u_k)_{i,:}+b^U_k,\quad
W^U_k\in\mathbb R^{r^k_1\times D},\; b^U_k\in\mathbb R^{r^k_1}.
\]
The same construction is used for \(V_k\) and \(W_k\). The head parameters are lightweight, and there are \(3K\) such heads in total [2508.12261].

## 3. Objective function and training procedure

The optimization variable is
\[
\Theta=
\{\Theta_{\rm shared}\}\cup
\bigl\{\Theta^U_k,\Theta^V_k,\Theta^W_k,\mathcal{C}_k\bigr\}_{k=1}^K,
\]
and training minimizes the mean-squared error over the observed entries:
\[
\min_{\Theta}\;
\sum_{k=1}^{K}\sum_{(i,j,\ell)\in\Omega_k}
\left\|
\bigl[\mathcal{C}_k\times_1U_k\times_2V_k\times_3W_k\bigr]_{i,j,\ell}
-\mathcal{T}_{i,j,\ell}
\right\|^2.
\]
No additional regularizers or explicit rank constraints are imposed; the low-rank structure is enforced by choosing small \(r^k_1,r^k_2,r^k_3\) and by the network bottleneck [2508.12261].

Training uses Adam with cosine-annealed learning rate. The reported schedules are: MSIs, 16 k iterations, initial LR in \([5\times10^{-5},5\times10^{-3}]\), weight decay in \([0.5,3]\), and SIREN frequency \(\omega_0\in[1,5]\); videos, 4 k iterations; RGB images, 3 k iterations.

The procedural pipeline is also specified. One first obtains a coarse guide via HaLRTC, applies SLIC, and gets superpixels \(\{\Omega_k\}\). Parameters are initialized, coordinate arrays \(C^u_k\), \(C^v_k\), \(C^w_k\) are sampled, embeddings are computed by \(\Phi\), and the factors \(U_k\), \(V_k\), \(W_k\) are generated by the corresponding heads. Each patch is reconstructed as
\[
X_k=C_k\times_1U_k\times_2V_k\times_3W_k,
\]
the loss is accumulated as \(L=\sum_k mse_k\), where \(mse_k=\|P_{\Omega_k}(X_k-T_k)\|^2\), and \(\Theta\) is updated by Adam. The resulting model is summarized in the source as a hybrid of Tucker decomposition and Implicit Neural Representation [2508.12261].

## 4. Relation to classical tensor factorization

The immediate baseline for ALTF is classical Tucker or CP factorization. The 2025 comparison is stated in four dimensions. First, global versus local: classical Tucker/CP assume a single set of factor matrices \(U,V,W\) are valid everywhere, whereas ALTF uses local factor matrices \(U_k,V_k,W_k\) per superpixel. Second, discrete versus continuous: traditional methods only model on fixed grid indices, while ALTF, via \(\Phi\), defines a continuous map \((u,v,w)\mapsto\text{features}\mapsto\hat{\mathcal{X}}(u,v,w)\), so one can query off-grid. Third, parameter efficiency via asymmetry: a naive scheme with \(K\) independent Tucker decompositions costs \(\mathcal O(K(H_kW_kL_k\,r_k^3))\) parameters, whereas ALTF uses one shared backbone, \(3K\) small heads, and \(K\) small cores. Fourth, divide-and-conquer: superpixels break spatial heterogeneity into simpler low-rank pieces, and each rank \(r^k_i\) can be very small \((1\!-\!3)\) [2508.12261].

Earlier work on nonsymmetric low-rank tensor approximation provides a distinct but closely related perspective. In the low-rank tensor approximation problem, one minimizes \(\|A-\hat A\|_F\) subject to \(\operatorname{rank}(\hat A)\le r\), with \(\hat A\) written in canonical polyadic form
\[
\hat A=\sum_{s=1}^r u_s^{(1)}\otimes u_s^{(2)}\otimes\cdots\otimes u_s^{(d)}.
\]
The 2014 method proceeds in three stages: it finds approximately satisfied linear relations expressed by polynomials and solved by linear least squares; computes points that are approximately common zeros of the obtained polynomials by computing Schur decompositions; and constructs a low-rank approximating tensor from the obtained points by solving linear least squares [1412.7270].

A second line of work casts asymmetric tensor factorization as CP tensor factorization via random projections and simultaneous matrix diagonalization. For a third-order tensor \(T\in\mathbb R^{n_1\times n_2\times n_3}\), the decomposition
\[
T=\sum_{r=1}^R \lambda_r\,a_r^{(1)}\circ a_r^{(2)}\circ a_r^{(3)}
\]
is treated without assuming orthogonality of the columns of the factor matrices. The algorithm uses random Gaussian projection matrices, mode-\(k\) matricizations, projected matrices \(M^{(k)}\), and a joint diagonalization problem
\[
\min_{Q\ \mathrm{orthonormal}}\sum_{j=1}^m
\|\,\mathrm{off}(Q^\top M_j^{(k)}Q)\|_F^2,
\]
followed by recovery of the weights \(\hat\lambda\) via ordinary least squares [1501.07320].

This suggests that the term “asymmetric” is used in more than one sense across the literature represented here. In the 2025 neural formulation it denotes a split between one global backbone and many local heads; in the earlier CP-oriented formulations it refers to nonsymmetric tensors or non-orthogonal factors [2508.12261].

## 5. Guarantees, computational structure, and algorithmic contrasts

The 2014 low-rank approximation analysis assumes
\[
A=X_*+E,\quad \operatorname{rank}(X_*)=r,\quad \|E\|_F=\epsilon,
\]
together with a mild set of nondegeneracy conditions, including genericity of the shared zeros, full-rank of the linear blocks \(A[A,T]\), and distinctness of weighted eigenvalues. Under these conditions,
\[
\|X_*-\hat A_{gp}\|_F=O(\epsilon),\qquad
\|A-\hat A_{gp}\|_F=O(\epsilon),
\]
and in particular
\[
\|A-\hat A_{gp}\|_F\le C\cdot \operatorname{dist}(A,\text{set of rank-}r\text{ tensors}),
\]
with \(C=O(1)\). A final nonlinear refinement can only reduce the residual further [1412.7270].

The same work gives a staged complexity estimate. Let \(N=\prod_{k=1}^d n_k\) and \(n_{\max}=\max_k n_k\). Stage 1, polynomial fitting and least squares in \(G\), has cost \(O((d-1)\,r^2\,(N/n_{\max}))\). Stage 2, Schur decomposition, has cost \(O(r^3)\). Stage 3, the final least squares for \(U^{(1)}\), has cost \(O(r^2\,(N/n_1)+r^3)\). Overall the complexity is \(\ll O(Nr^2+r^3)\), typically dominated by \(O(Nr^2)\) [1412.7270].

The 2015 matrix-factorization route gives a different guarantee. If the true factors are \(\{A^{(k)},\lambda\}\) and one takes
\[
m=O(\epsilon^{-2}\log(n_k/\delta)),
\]
then with probability at least \(1-\delta\), the joint diagonalizer yields
\[
\|\hat A^{(k)}-A^{(k)}P_k\|_2\le \epsilon
\]
for some signed permutation \(P_k\). The error bound depends on the noise magnitude \(\|E\|\) and on \(\epsilon\), but does not degrade as the minimum gap between any two \(\lambda_r\) goes to zero. Its total time is
\[
O\!\left(mN+\sum_{k=1}^3(n_k^3+n_k^2m)\right)+O(R^3),
\]
where \(N=\|T\|_0\) for sparse tensors or \(N=n_1n_2n_3\) for dense tensors [1501.07320].

Within the 2025 ALTF framework, the principal computational argument is not a closed-form global complexity bound but a structural one: one shared backbone, \(3K\) small heads, and \(K\) small cores replace a naive collection of independent Tucker models, while the bottleneck and small ranks enforce the low-rank structure [2508.12261].

## 6. Empirical behavior and interpretive issues

The 2025 experiments are reported on multispectral images from CAVE of size \(256\times256\times31\), videos from the ASU library of size \(144\times176\times900\), and color images from USC-SIPI of size \(256\times256\times3\), with sampling rates from \(5\%\) to \(25\%\). The metrics are PSNR and SSIM. The abstract states that SCTR achieves \(3\!-\!5\) dB PSNR improvements over existing LRTR-based methods across multispectral images, videos, and color images [2508.12261].

| Setting | Baseline result | SCTR result |
|---|---:|---:|
| MSI, 15% sampling | LRTFR: 44.65 dB, 0.9595 | 48.38 dB, 0.9943 |
| Video, 10% sampling | FCTN: 29.14 dB, 0.8118 | 30.81 dB, 0.8906 |
| RGB, 15% sampling | t-CTV: 24.76 dB, 0.7906 | 25.25 dB, 0.8029 |

For the MSI case at 15% sampling, the gain over the second best LRTFR is \(+3.73\) dB. For videos at 10% sampling, the gain is \(+1.67\) dB and \(+0.0788\) SSIM. The ablation study at MSI 10% sampling reports: no superpixel and no ALTF, 39.11 dB; no superpixel, with ALTF, 42.12 dB; with superpixel, no ALTF, 43.49 dB; full SCTR, 46.66 dB. The stated conclusion is that both superpixel partitioning and the asymmetric network factorization are essential [2508.12261].

The 2014 experiments on random nearly low-rank tensors report that for random \(A=X_*+E\) with \(\|E\|/\|X_*\|\approx10^{-k}\), \(k=1,2,3\), the method recovers \(\hat A_{gp}\) with \(\|A-\hat A_{gp}\|/\|E\|=O(1)\), in perfect agreement with the \(O(\epsilon)\) bound. Runtimes grow roughly linearly with the tensor size in each mode and quadratically in \(r\). When \(\operatorname{rank}(A)=r\) exactly, the recovered \(\hat A_{gp}\) is an exact decomposition up to \(10^{-10}\) round-off, and no further refinement is needed. Compared with classical ALS, it converges in one shot, is insensitive to poor initialization, and can handle larger dimensions, including modes of size up to 100 in modest time on a laptop [1412.7270].

The 2015 experiments report synthetic tests with random \(A^{(k)}\), \(R=10\!-\!50\), \(n_k\) up to 500, and noise up to SNR \(=10\) dB, where the method recovers factors with \(\|\hat A-A\|/\|A\|<10^{-3}\) using only \(m=O(50)\) projections. The same study reports applications to topic modeling on NIPS abstracts and collaborative filtering with a user\(\times\)movie\(\times\)time tensor, where the method reaches comparable or better held-out likelihood in a fraction of ALS runtime and avoids local minima [1501.07320].

A recurrent misconception is to treat ALTF as merely a collection of independent local tensor decompositions. The formulation in SCTR explicitly rejects that interpretation: the factor matrices are patch-specific, but their generation is amortized by a shared backbone, and the resulting asymmetry is the mechanism by which global pattern learning is separated from local adaptation. A second source of confusion is terminological: earlier tensor-factorization papers use “asymmetric” to denote non-orthogonal or nonsymmetric CP structure, whereas the 2025 work uses it to describe the architecture of the factor-generation pathway.

Source: https://www.emergentmind.com/topics/asymmetric-low-rank-tensor-factorization-altf