Asymmetric Low-rank Tensor Factorization (ALTF)

• Asymmetric Low-rank Tensor Factorization (ALTF) is a method that decomposes incomplete multi-dimensional data into superpixel-specific Tucker models using a shared neural backbone and local factor heads.
• It enhances efficiency by splitting the factor-generation pathway into one global module and multiple localized heads, significantly reducing parameter overhead compared to full Tucker decompositions.
• Empirical results show that ALTF achieves notable gains in PSNR and SSIM on multispectral images, videos, and color images, validating its superior reconstruction performance.

Searching arXiv for the cited papers and related ALTF usage.

arxiv_search query: "(Nie, 2014) Nearly Low Rank Tensors and Their Approximations"

Asymmetric Low-rank Tensor Factorization (ALTF) denotes, in the superpixel-informed continuous low-rank tensor representation framework, a low-rank tensor model for incomplete multi-dimensional data in which each superpixel receives its own Tucker core and factor matrices, while the factor matrices are generated through a shared neural backbone together with patch-specific heads. In this formulation, ALTF is explicitly asymmetric because the factor-generation path is split between one global module and multiple local modules. Related earlier work studies nonsymmetric or asymmetric low-rank tensor approximation through CP-style decompositions, polynomial relations, Schur decompositions, and random-projection-based simultaneous matrix diagonalization, placing ALTF within the broader landscape of low-rank tensor approximation problems and tensor factorization algorithms (Wang et al., 17 Aug 2025).

1. Formal definition and tensor model

The 2025 ALTF formulation considers an incomplete 3-D tensor $\mathcal{T}\in\mathbb{R}^{I\times J\times K}$ together with a partition of its spatial indices into $K$ superpixels, each patch indexed by $k=1,\dots,K$. For each superpixel $k$, the observed entries are denoted by $$\Omega_k\subset\{1,\dots,I\}\times\{1,\dots,J\}\times\{1,\dots,K\}$$. As a point of reference, classical Tucker decomposition is written as

$$\mathcal{X}\approx\mathcal{C}\times_{1}U\times_{2}V\times_{3}W$$

with $U\in\mathbb R^{I\times r_1}$, $V\in\mathbb R^{J\times r_2}$, and $W\in\mathbb R^{K\times r_3}$.

ALTF breaks the symmetry of “one global core + three global factors” by assigning each superpixel $k$ its own small core $K$0 and factor matrices $K$1, $K$2, $K$3, while amortizing their generation via a shared backbone network plus patch-specific heads. The coordinate arrays for patch $K$4 are

$K$5

where $K$6 is the spatial size of superpixel $K$7.

The reconstructed patch is a Tucker composition,

$K$8

The defining asymmetry is described as a split between a single global MLP $K$9, which captures shared spectral/spatial coordinate priors, and multiple local heads $k=1,\dots,K$0, each adapting to one superpixel (Wang et al., 17 Aug 2025).

2. Neural parameterization of the factor matrices

The shared backbone is an MLP

$k=1,\dots,K$1

which maps any 1-D coordinate array to a $k=1,\dots,K$2-dimensional embedding per coordinate. Figure 1 is summarized as follows: the backbone takes as input a vector of $k=1,\dots,K$3 sorted 1-D coordinates; it contains a SineLayer, defined as a linear layer $k=1,\dots,K$4 followed by $k=1,\dots,K$5; four ResidualSineBlocks, each with two linear layers, sine activations, and a skip-connection; then a ReLU; and finally an AttentionBlock projecting to a $k=1,\dots,K$6-dimensional per-coordinate feature (Wang et al., 17 Aug 2025).

For each patch $k=1,\dots,K$7, there are three lightweight head-networks,

$k=1,\dots,K$8

which transform the shared embedding into the factor matrices

$k=1,\dots,K$9

Each core $k$0 is a small, learnable tensor.

The head $k$1 is a single linear layer per superpixel mapping $k$2, so each row of $k$3 is

$k$4

The same construction is used for $k$5 and $k$6. The head parameters are lightweight, and there are $k$7 such heads in total (Wang et al., 17 Aug 2025).

3. Objective function and training procedure

The optimization variable is

$k$8

and training minimizes the mean-squared error over the observed entries: $k$9 No additional regularizers or explicit rank constraints are imposed; the low-rank structure is enforced by choosing small $$\Omega_k\subset\{1,\dots,I\}\times\{1,\dots,J\}\times\{1,\dots,K\}$$0 and by the network bottleneck (Wang et al., 17 Aug 2025).

Training uses Adam with cosine-annealed learning rate. The reported schedules are: MSIs, 16 k iterations, initial LR in $$\Omega_k\subset\{1,\dots,I\}\times\{1,\dots,J\}\times\{1,\dots,K\}$$1, weight decay in $$\Omega_k\subset\{1,\dots,I\}\times\{1,\dots,J\}\times\{1,\dots,K\}$$2, and SIREN frequency $$\Omega_k\subset\{1,\dots,I\}\times\{1,\dots,J\}\times\{1,\dots,K\}$$3; videos, 4 k iterations; RGB images, 3 k iterations.

The procedural pipeline is also specified. One first obtains a coarse guide via HaLRTC, applies SLIC, and gets superpixels $$\Omega_k\subset\{1,\dots,I\}\times\{1,\dots,J\}\times\{1,\dots,K\}$$4. Parameters are initialized, coordinate arrays $$\Omega_k\subset\{1,\dots,I\}\times\{1,\dots,J\}\times\{1,\dots,K\}$$5, $$\Omega_k\subset\{1,\dots,I\}\times\{1,\dots,J\}\times\{1,\dots,K\}$$6, $$\Omega_k\subset\{1,\dots,I\}\times\{1,\dots,J\}\times\{1,\dots,K\}$$7 are sampled, embeddings are computed by $$\Omega_k\subset\{1,\dots,I\}\times\{1,\dots,J\}\times\{1,\dots,K\}$$8, and the factors $$\Omega_k\subset\{1,\dots,I\}\times\{1,\dots,J\}\times\{1,\dots,K\}$$9, $$\mathcal{X}\approx\mathcal{C}\times_{1}U\times_{2}V\times_{3}W$$0, $$\mathcal{X}\approx\mathcal{C}\times_{1}U\times_{2}V\times_{3}W$$1 are generated by the corresponding heads. Each patch is reconstructed as

$$\mathcal{X}\approx\mathcal{C}\times_{1}U\times_{2}V\times_{3}W$$2

the loss is accumulated as $$\mathcal{X}\approx\mathcal{C}\times_{1}U\times_{2}V\times_{3}W$$3, where $$\mathcal{X}\approx\mathcal{C}\times_{1}U\times_{2}V\times_{3}W$$4, and $$\mathcal{X}\approx\mathcal{C}\times_{1}U\times_{2}V\times_{3}W$$5 is updated by Adam. The resulting model is summarized in the source as a hybrid of Tucker decomposition and Implicit Neural Representation (Wang et al., 17 Aug 2025).

4. Relation to classical tensor factorization

The immediate baseline for ALTF is classical Tucker or CP factorization. The 2025 comparison is stated in four dimensions. First, global versus local: classical Tucker/CP assume a single set of factor matrices $$\mathcal{X}\approx\mathcal{C}\times_{1}U\times_{2}V\times_{3}W$$6 are valid everywhere, whereas ALTF uses local factor matrices $$\mathcal{X}\approx\mathcal{C}\times_{1}U\times_{2}V\times_{3}W$$7 per superpixel. Second, discrete versus continuous: traditional methods only model on fixed grid indices, while ALTF, via $$\mathcal{X}\approx\mathcal{C}\times_{1}U\times_{2}V\times_{3}W$$8, defines a continuous map $$\mathcal{X}\approx\mathcal{C}\times_{1}U\times_{2}V\times_{3}W$$9, so one can query off-grid. Third, parameter efficiency via asymmetry: a naive scheme with $U\in\mathbb R^{I\times r_1}$0 independent Tucker decompositions costs $U\in\mathbb R^{I\times r_1}$1 parameters, whereas ALTF uses one shared backbone, $U\in\mathbb R^{I\times r_1}$2 small heads, and $U\in\mathbb R^{I\times r_1}$3 small cores. Fourth, divide-and-conquer: superpixels break spatial heterogeneity into simpler low-rank pieces, and each rank $U\in\mathbb R^{I\times r_1}$4 can be very small $U\in\mathbb R^{I\times r_1}$5 (Wang et al., 17 Aug 2025).

Earlier work on nonsymmetric low-rank tensor approximation provides a distinct but closely related perspective. In the low-rank tensor approximation problem, one minimizes $U\in\mathbb R^{I\times r_1}$6 subject to $U\in\mathbb R^{I\times r_1}$7, with $U\in\mathbb R^{I\times r_1}$8 written in canonical polyadic form

$U\in\mathbb R^{I\times r_1}$9

The 2014 method proceeds in three stages: it finds approximately satisfied linear relations expressed by polynomials and solved by linear least squares; computes points that are approximately common zeros of the obtained polynomials by computing Schur decompositions; and constructs a low-rank approximating tensor from the obtained points by solving linear least squares (Nie, 2014).

A second line of work casts asymmetric tensor factorization as CP tensor factorization via random projections and simultaneous matrix diagonalization. For a third-order tensor $V\in\mathbb R^{J\times r_2}$0, the decomposition

$V\in\mathbb R^{J\times r_2}$1

is treated without assuming orthogonality of the columns of the factor matrices. The algorithm uses random Gaussian projection matrices, mode-$V\in\mathbb R^{J\times r_2}$2 matricizations, projected matrices $V\in\mathbb R^{J\times r_2}$3, and a joint diagonalization problem

$V\in\mathbb R^{J\times r_2}$4

followed by recovery of the weights $V\in\mathbb R^{J\times r_2}$5 via ordinary least squares (Kuleshov et al., 2015).

This suggests that the term “asymmetric” is used in more than one sense across the literature represented here. In the 2025 neural formulation it denotes a split between one global backbone and many local heads; in the earlier CP-oriented formulations it refers to nonsymmetric tensors or non-orthogonal factors (Wang et al., 17 Aug 2025).

5. Guarantees, computational structure, and algorithmic contrasts

The 2014 low-rank approximation analysis assumes

$V\in\mathbb R^{J\times r_2}$6

together with a mild set of nondegeneracy conditions, including genericity of the shared zeros, full-rank of the linear blocks $V\in\mathbb R^{J\times r_2}$7, and distinctness of weighted eigenvalues. Under these conditions,

$V\in\mathbb R^{J\times r_2}$8

and in particular

$V\in\mathbb R^{J\times r_2}$9

with $W\in\mathbb R^{K\times r_3}$0. A final nonlinear refinement can only reduce the residual further (Nie, 2014).

The same work gives a staged complexity estimate. Let $W\in\mathbb R^{K\times r_3}$1 and $W\in\mathbb R^{K\times r_3}$2. Stage 1, polynomial fitting and least squares in $W\in\mathbb R^{K\times r_3}$3, has cost $W\in\mathbb R^{K\times r_3}$4. Stage 2, Schur decomposition, has cost $W\in\mathbb R^{K\times r_3}$5. Stage 3, the final least squares for $W\in\mathbb R^{K\times r_3}$6, has cost $W\in\mathbb R^{K\times r_3}$7. Overall the complexity is $W\in\mathbb R^{K\times r_3}$8, typically dominated by $W\in\mathbb R^{K\times r_3}$9 (Nie, 2014).

The 2015 matrix-factorization route gives a different guarantee. If the true factors are $k$0 and one takes

$k$1

then with probability at least $k$2, the joint diagonalizer yields

$k$3

for some signed permutation $k$4. The error bound depends on the noise magnitude $k$5 and on $k$6, but does not degrade as the minimum gap between any two $k$7 goes to zero. Its total time is

$k$8

where $k$9 for sparse tensors or $K$00 for dense tensors (Kuleshov et al., 2015).

Within the 2025 ALTF framework, the principal computational argument is not a closed-form global complexity bound but a structural one: one shared backbone, $K$01 small heads, and $K$02 small cores replace a naive collection of independent Tucker models, while the bottleneck and small ranks enforce the low-rank structure (Wang et al., 17 Aug 2025).

6. Empirical behavior and interpretive issues

The 2025 experiments are reported on multispectral images from CAVE of size $K$03, videos from the ASU library of size $K$04, and color images from USC-SIPI of size $K$05, with sampling rates from $K$06 to $K$07. The metrics are PSNR and SSIM. The abstract states that SCTR achieves $K$08 dB PSNR improvements over existing LRTR-based methods across multispectral images, videos, and color images (Wang et al., 17 Aug 2025).

Setting	Baseline result	SCTR result
MSI, 15% sampling	LRTFR: 44.65 dB, 0.9595	48.38 dB, 0.9943
Video, 10% sampling	FCTN: 29.14 dB, 0.8118	30.81 dB, 0.8906
RGB, 15% sampling	t-CTV: 24.76 dB, 0.7906	25.25 dB, 0.8029

For the MSI case at 15% sampling, the gain over the second best LRTFR is $K$09 dB. For videos at 10% sampling, the gain is $K$10 dB and $K$11 SSIM. The ablation study at MSI 10% sampling reports: no superpixel and no ALTF, 39.11 dB; no superpixel, with ALTF, 42.12 dB; with superpixel, no ALTF, 43.49 dB; full SCTR, 46.66 dB. The stated conclusion is that both superpixel partitioning and the asymmetric network factorization are essential (Wang et al., 17 Aug 2025).

The 2014 experiments on random nearly low-rank tensors report that for random $K$12 with $K$13, $K$14, the method recovers $K$15 with $K$16, in perfect agreement with the $K$17 bound. Runtimes grow roughly linearly with the tensor size in each mode and quadratically in $K$18. When $K$19 exactly, the recovered $K$20 is an exact decomposition up to $K$21 round-off, and no further refinement is needed. Compared with classical ALS, it converges in one shot, is insensitive to poor initialization, and can handle larger dimensions, including modes of size up to 100 in modest time on a laptop (Nie, 2014).

The 2015 experiments report synthetic tests with random $K$22, $K$23, $K$24 up to 500, and noise up to SNR $K$25 dB, where the method recovers factors with $K$26 using only $K$27 projections. The same study reports applications to topic modeling on NIPS abstracts and collaborative filtering with a user$K$28movie$K$29time tensor, where the method reaches comparable or better held-out likelihood in a fraction of ALS runtime and avoids local minima (Kuleshov et al., 2015).

A recurrent misconception is to treat ALTF as merely a collection of independent local tensor decompositions. The formulation in SCTR explicitly rejects that interpretation: the factor matrices are patch-specific, but their generation is amortized by a shared backbone, and the resulting asymmetry is the mechanism by which global pattern learning is separated from local adaptation. A second source of confusion is terminological: earlier tensor-factorization papers use “asymmetric” to denote non-orthogonal or nonsymmetric CP structure, whereas the 2025 work uses it to describe the architecture of the factor-generation pathway.