Smooth Tensor Decomposition

• Smooth tensor decomposition is a framework that embeds smoothness constraints (e.g., generalized-lasso, total variation, RKHS) directly into tensor factorization for ordered data.
• It utilizes various regularization techniques and geometric formulations to ensure continuous, stable latent factors, enhancing model robustness and interpretability.
• Applications include improved tensor completion, medical imaging analysis, and temporal knowledge graph embedding by leveraging smooth reconstructions.

Searching arXiv for papers directly relevant to smooth tensor decomposition and neighboring formulations. arXiv search: "smooth tensor decomposition" Smooth tensor decomposition denotes a family of tensor factorization frameworks in which smoothness, regularity, or differentiable geometry is built into the decomposition itself rather than treated as an external post-processing step. In the literature, this designation covers several non-equivalent constructions: low-rank models with generalized-lasso penalties on ordered factor vectors, CP models with total-variation or quadratic-variation regularization on latent components, Tucker-type models that penalize the smoothness of the reconstructed trajectories, hybrid finite/infinite-dimensional decompositions with RKHS-valued factors, information-geometric decompositions on statistical manifolds, and fixed-rank normalized tensor families that form smooth manifolds under unit-norm constraints (Padilla et al., 2015, Yokota et al., 2015, Qian et al., 15 Jul 2025, Larsen et al., 2024, Sugiyama et al., 2018, Peng et al., 6 Nov 2025). The common theme is structural regularity: temporal, spatial, probabilistic, or geometric constraints are imposed so that the recovered factors or fitted tensor respect properties that ordinary CP, Tucker, or TT models do not encode intrinsically.

1. Conceptual scope

The term is not monolithic. In one line of work, smoothness means that latent factors vary gradually along an ordered mode. Penalized tensor decomposition (PTD) introduces generalized-lasso penalties such as first differences, higher-order trend filtering, and graph incidence operators on the factor vectors themselves, so that the factors are piecewise smooth, locally flat, or smoothly varying in space or time (Padilla et al., 2015). Smooth PARAFAC tensor completion (SPC) likewise regularizes the latent CP component vectors with finite-difference penalties, using either total variation or quadratic variation (Yokota et al., 2015).

In a second line, smoothness is attached not to the factor matrix but to the fitted signal. SmoothHOOI, developed for ambulatory blood pressure monitoring data, penalizes the second-difference roughness of the reconstructed slices

$\widehat M_i = L G_i R^\top,$

rather than penalizing $L$ directly (Qian et al., 15 Jul 2025). This distinction is central in that work: direct smoothing of $L$ under orthogonality constraints can align its columns with eigenvectors of $D^\top D$, whereas smoothing the reconstruction targets the clinically expected property that the observed trajectories themselves are smooth.

A third usage treats one or more tensor modes as genuinely continuous. CP-HiFi replaces selected CP factor vectors by RKHS functions, producing a decomposition of a quasitensor rather than a fully discrete tensor, and thereby enforcing smoothness in the continuous modes while accommodating irregular or misaligned observations (Larsen et al., 2024). A fourth usage is geometric: Legendre decomposition places normalized positive tensors in a dually flat statistical manifold, while normalized tensor train (NTT) decomposition defines the fixed-rank, unit-Frobenius-norm TT set as a smooth manifold (Sugiyama et al., 2018, Peng et al., 6 Nov 2025). These geometric formulations are smooth in the differential-geometric sense rather than only in the signal-regularization sense.

2. Penalized low-rank decompositions for ordered data

PTD extends Parafac/CANDECOMP by imposing generalized-lasso penalties on factor vectors. In the rank-1 constrained form, it maximizes

$$\underline{Y} \times_{1} u \times_{2} v \times_{3} w$$

subject to $\|D^{u}u\|_1 \le c_u$, $\|D^{v}v\|_1 \le c_v$, $\|D^{w}w\|_1 \le c_w$, and $\ell_2$-norm constraints on the factors (Padilla et al., 2015). Choosing $D$ as a first-difference matrix yields fused-lasso behavior and piecewise-constant factors; choosing trend-filtering operators yields piecewise polynomial factors; choosing a graph incidence matrix yields graph-structured smoothness. The unconstrained formulation replaces the constraints by penalties $L$0, which the paper reports to be much more computationally efficient in practice.

SPC imposes smoothness directly on each CP component vector $L$1 through

$L$2

with $L$3 for SPC-TV and $L$4 for SPC-QV (Yokota et al., 2015). The factor $L$5 adaptively weights the regularization by component magnitude. Unlike methods that smooth the reconstructed tensor surface, SPC regularizes the latent component vectors themselves. Its model-selection procedure is rank-increasing rather than rank-decreasing: the algorithm starts at $L$6 and adds components until the observed-entry error falls below a target threshold derived from a desired SDR. The paper argues that this is advantageous because the effective rank of a smooth CP model can exceed that of an unconstrained CP model.

SmoothHOOI adopts a Tucker/GLRAM parameterization tailored to a tensor $L$7 with time, measurement type, and patient modes. With the patient mode left uncompressed, the objective becomes

$L$8

subject to $L$9 and $L$0 (Qian et al., 15 Jul 2025). This formulation explicitly accommodates missing data through $L$1 and enforces smooth reconstructed temporal profiles through the second-difference operator $L$2. The design suggests a separation between low-dimensional latent structure and regularity of the fitted trajectories.

3. Continuous-mode and information-geometric formulations

CP-HiFi generalizes CP decomposition by allowing some modes to be infinite-dimensional. For a three-way quasitensor, the model is

$L$3

where $L$4 lies in an RKHS (Larsen et al., 2024). By the representer theorem, each continuous factor has the form

$L$5

so the infinite-dimensional subproblem reduces to a finite-dimensional kernel problem. The resulting optimization combines a masked least-squares fit with an RKHS penalty $L$6. Because the loss is evaluated only on observed entries, the method does not require the data to lie on a regular finite rectangular grid and naturally incorporates misaligned data. In the synthetic study reported in the paper, ordinary CP and CP-HiFi behave similarly on aligned dense samples, whereas CP-HiFi remains smoother and more stable when samples are sparse or unaligned.

Legendre decomposition takes a different route by treating a normalized nonnegative tensor as a discrete probability distribution. Starting from $L$7, it defines

$L$8

and reconstructs

$L$9

on a partially ordered index set (Sugiyama et al., 2018). The model is an exponential-family-like smooth factorization with dual affine coordinates $D^\top D$0, where $D^\top D$1. The optimal reconstruction is the unique KL-best approximation in the decomposable family:

$D^\top D$2

The gradient is $D^\top D$3, so the optimum satisfies moment matching $D^\top D$4 for $D^\top D$5. Because $D^\top D$6 and the KL objective are convex in $D^\top D$7, the optimization is globally well behaved, and the paper notes that the natural gradient coincides with Newton’s method because the Fisher information matrix equals the negative Hessian.

4. Smooth manifolds and constrained tensor families

NTT decomposition is a geometry-preserving variant of tensor train decomposition for tensors with unit Frobenius norm. It keeps the usual TT core structure

$D^\top D$8

but adds the constraint

$D^\top D$9

The fixed-rank NTT set is

$$\underline{Y} \times_{1} u \times_{2} v \times_{3} w$$0

and the paper proves that $$\underline{Y} \times_{1} u \times_{2} v \times_{3} w$$1 is a smooth manifold because $$\underline{Y} \times_{1} u \times_{2} v \times_{3} w$$2 and the unit sphere intersect transversally (Peng et al., 6 Nov 2025). This permits explicit tangent-space formulas, tangent projections obtained by composing TT tangent projection with sphere projection, and a retraction implemented by TT-SVD followed by normalization. The quasi-optimality bound

$$\underline{Y} \times_{1} u \times_{2} v \times_{3} w$$3

shows that the practical truncation operator remains close to the metric projection. The corresponding NTT-RCG algorithm is then used for low-rank tensor recovery, high-dimensional eigenvalue problems, approximate stabilizer rank, and minimum output Rényi $$\underline{Y} \times_{1} u \times_{2} v \times_{3} w$$4-entropy computations.

A related but application-specific geometric construction appears in temporal knowledge graph embedding. There, factor tensors for entities, relations, objects, and timestamps are mapped onto a unified smooth Lie group manifold, specifically $$\underline{Y} \times_{1} u \times_{2} v \times_{3} w$$5, to mitigate heterogeneity among factor tensors in tensor-decomposition-based models (Li et al., 2024). The paper defines $$\underline{Y} \times_{1} u \times_{2} v \times_{3} w$$6 by sending an embedding $$\underline{Y} \times_{1} u \times_{2} v \times_{3} w$$7 to a rotation matrix, applies the logarithmic map $$\underline{Y} \times_{1} u \times_{2} v \times_{3} w$$8, and forms transformed factors such as

$$\underline{Y} \times_{1} u \times_{2} v \times_{3} w$$9

The stated motivation is that homogeneous factor tensors can approximate the target tensor with lower rank than heterogeneous factor tensors. This is not a new low-rank family in the CP or Tucker sense, but it is a smooth-manifold regularization mechanism for tensor-decomposition-based TKGE.

5. Optimization, identifiability, and stability

The algorithmic landscape is correspondingly heterogeneous. PTD uses block coordinate descent in which each factor update is a generalized-lasso subproblem, solved either through a constrained dual characterization or through an unconstrained generalized-lasso regression followed by normalization (Padilla et al., 2015). SPC uses HALS-style block updates of the rank-one terms together with a rank-increasing outer loop (Yokota et al., 2015). CP-HiFi uses alternating optimization over discrete factor matrices and RKHS weight matrices, with continuous-mode updates reducing to linear systems in the kernel coefficients (Larsen et al., 2024). SmoothHOOI alternates between eigendecomposition-based updates for the temporal and measurement subspaces, closed-form updates for $\|D^{u}u\|_1 \le c_u$0, and iterative imputation of missing entries (Qian et al., 15 Jul 2025). NTT uses Riemannian conjugate gradient on $\|D^{u}u\|_1 \le c_u$1, with tangent projection, transported directions, and NTT-SVD retraction (Peng et al., 6 Nov 2025).

Uniqueness and interpretability also differ across frameworks. Legendre decomposition provides the strongest global statement: the reconstructed tensor is the unique global KL minimizer in the decomposable family when the basis and target constraints align appropriately (Sugiyama et al., 2018). SmoothHOOI inherits Tucker rotational non-uniqueness, so the paper applies an SVD-based post-rotation to the core unfoldings in order to order components by explained variability and make them interpretable (Qian et al., 15 Jul 2025). NTT does not claim a uniqueness theorem of that type, but its fixed-rank constraint set has a well-defined smooth geometry and quasi-optimal truncation theory (Peng et al., 6 Nov 2025).

A separate notion of smoothness appears in noise stability. Two-mode HOSVD for nearly orthogonally decomposable symmetric tensors characterizes the true factors through the left singular space of a two-mode unfolding and proves perturbation bounds for the recovered components (Wang et al., 2016). The reported bounds do not depend on eigengaps, and the post-processing step improves the factor error from a dimension-dependent bound to $\|D^{u}u\|_1 \le c_u$2. This is not a smooth-manifold formulation, but it exemplifies a continuity-based reading of smooth tensor decomposition: the decomposition changes stably under perturbation.

6. Applications, empirical patterns, and related but distinct notions

The application profile is broad but structurally consistent. PTD is aimed at multiway data with ordered modes such as temporal, spatial, or graph-structured variation; its simulations show that fused lasso is best for piecewise constant factors, trend filtering is best for smoothly varying or polynomial-like factors, and plain $\|D^{u}u\|_1 \le c_u$3 penalties are preferable when the truth is only sparse (Padilla et al., 2015). SPC is designed for tensor completion under extremely high missingness, especially visual data; the paper reports strong performance on synthetic tensors, color images, MRI data, and CMU faces, with SPC-QV generally outperforming SPC-TV and other completion baselines at high missing rates (Yokota et al., 2015). CP-HiFi targets smooth and misaligned data, where a continuous mode is sampled irregularly across fibers, and its synthetic example shows that RKHS factors remain faithful when ordinary CP becomes jagged or distorted (Larsen et al., 2024).

SmoothHOOI is specialized to partially observed multivariate longitudinal tensors. Applied to a $\|D^{u}u\|_1 \le c_u$4 ABPM tensor from the HYPNOS study, it decomposes the data into interpretable temporal components corresponding to overall level, nocturnal dipping, and sleep timing or chronotype shift, together with measurement components that separate weighted SBP/DBP/HR averages from BP-versus-HR contrast (Qian et al., 15 Jul 2025). The paper reports that higher ODI4 is significantly associated with higher overall BP/HR level, a relation that was missed by the summary-statistic analysis. NTT addresses normalized high-dimensional tensors in scientific computing and quantum information, while the Lie-group TKGE method reports reduced heterogeneity distances and improved MRR and Hits@$\|D^{u}u\|_1 \le c_u$5 on ICEWS14, ICEWS05-15, and GDELT without adding parameters (Peng et al., 6 Nov 2025, Li et al., 2024).

The term must also be distinguished from neighboring usages. “Smoothed analysis of tensor decompositions” studies random perturbations of tensor components that make overcomplete decomposition stable and efficient; it is a beyond-worst-case framework, not a smoothness-regularized factorization model (Bhaskara et al., 2013, Ma et al., 2016). “Decomposition of linear tensor transformations” develops an exact algebraic spectral framework based on vector-to-linear index transformations and SA-NND operators; the paper explicitly does not present a smooth tensor decomposition theory in the differentiable or manifold sense (Turchetti, 2023). These distinctions matter because the modern literature uses “smooth” to denote regularized signal structure, differentiable geometry, or perturbation stability, and only some of these are directly comparable.