NeuTucker Decomposition Model

• The NeuTucker model is a multilinear, low-rank framework that extends traditional Tucker decomposition with neural, nonnegative, and supervised techniques.
• It employs quantile-based discretization, embedding layers, and specialized optimization methods for computational efficiency and robust latent structure discovery.
• Applied in wind field regression, neuroimaging, and multilayer network analysis, it consistently achieves lower error metrics and enhanced interpretability.

The NeuTucker decomposition model is a multilinear, low-rank framework for tensor analysis and regression, generalizing classical Tucker decomposition into neural, nonnegative, and supervised domains. It is designed to model high-order, nonlinear interactions in multidimensional, often sparse or nonnegative, tensor data—such as wind-field measurements, neuroimaging arrays, multilayer networks, and recursively structured neural architectures—while maintaining computational tractability, interpretability, and statistical consistency. NeuTucker models leverage core ideas from multilinear algebra, embedding layers, quantile-based discretization, EM or gradient optimization, and regularization to achieve robust estimation and latent structure discovery.

1. Mathematical Foundations and Model Structure

NeuTucker models build on the Tucker decomposition, which expresses an order-$D$ tensor $$\mathcal{Y}\in\mathbb{R}^{I_1\times\cdots\times I_D}$$ as $$\mathcal{Y}\approx\mathcal{G}\times_1 A^{(1)}\times_2\cdots\times_D A^{(D)}$$, where $\mathcal{G}$ is a low-rank core tensor and $A^{(d)}$ are factor matrices along each mode. This multilinear framework generalizes matrix factorization concepts and enables powerful dimensionality reduction, capturing cross-mode interactions.

In the supervised and neural extension, such as for wind-field regression (Fan et al., 5 Dec 2025), the decomposition is embedded into a neural network that maps discrete indices of observed features $(p, i, j, k)$ to latent embeddings $e^{(p)}, e^{(i)}, e^{(j)}, e^{(k)}$. These vectors are combined via a 4-way outer product to yield the full Tucker interaction tensor, vectorized and passed through a linear layer with learnable weights (equivalent to flattening $\mathcal{G}$), followed by a nonlinear activation (e.g., sigmoid) producing the scalar prediction.

For nonnegative and network settings, such as multilayer graphs (Aguiar et al., 2022, Zhou et al., 2014), the core tensor and factors are constrained to be nonnegative, with the objective typically a KL divergence between observed and reconstructed edge-count tensors. Alternating block-coordinate or multiplicative updates optimize the solution, and identifiability, uniqueness, and sparsity are formally addressed.

2. Data Discretization and Embedding Strategies

Handling continuous input data in the NeuTucker model requires discretization suited for sparse, high-dimensional arrays:

• Standardization: Each continuous feature $X$ is standardized via $X_\text{std} = (X-\mu)/\sigma$, with $\mu$, $\sigma$ the empirical mean and standard deviation (Fan et al., 5 Dec 2025).
• Quantile-based binning: Discrete bins are defined using $K$ quantile boundaries $B_k=Q(k/K)$, where $Q(p)$ is the $p$-th sample quantile.
• Index assignment: Scalars $x$ are mapped to integer indices according to bin boundaries, yielding index vectors and tensors.
• Vector- and tensor-wise discretization: This process generalizes for arbitrary modes to obtain an integer-valued tensor $\mathcal{Z}$ suitable for embedding lookup.

Embeddings $E$ are layer parameters transforming integer indices into latent vectors of fixed dimension; these vector representations form the basis for the multilinear interaction tensor.

3. Training Objectives and Optimization Methods

NeuTucker models employ losses tailored to the application:

• Reconstruction loss: Typically mean-squared error (MSE) for regression; for network and nonnegative tensor settings, generalized KL divergence between observed and reconstructed tensors is used (Aguiar et al., 2022, Zhou et al., 2014).
• Regularization: $\ell_2$ penalties on embedding matrices and the core enhance stability and prevent overfitting; $\ell_1$ penalties enforce sparsity, improving uniqueness, interpretability, and scaling (Zhou et al., 2014).
• Block-coordinate descent: Alternating update of factors and core, with algorithms such as multiplicative updates (MU), hierarchical ALS (HALS), and accelerated proximal gradient (APG); for KL objectives, EM updates are algebraically identical to MUs.

Hyperparameter selection (embedding dimension, Tucker ranks, regularization strengths) is performed via grid search and cross-validation. For neural wind-field regression (Fan et al., 5 Dec 2025), Adam optimizer is used with early stopping by validation MAE.

4. Applications in Scientific Domains

NeuTucker decomposition underpins diverse scientific applications:

• Wind field regression: In sparse, continuous, 3D wind field datasets, discretized NeuTucker models outperform MLPs, linear, pairwise, and deep pairwise regression, achieving lowest error metrics and highest R² (Fan et al., 5 Dec 2025).
• Neuroimaging tensor regression: Multimodal image arrays (EEG, MRI, fMRI) are modeled as covariates in generalized linear models, with Tucker decomposition imposed on the coefficient tensor for dimensionality reduction and interpretability. Block-coordinate IRLS enables tractable estimation, and empirical studies show Tucker-based methods recover more complex signals than CP models (Li et al., 2013).
• Multilayer network analysis: NNTuck models provide a unified factorization for multi-relationship graphs, incorporating layer interdependence, redundancy, or independence. Layer-group structure is inferred by varying the third-mode rank and employing likelihood ratio tests. Extensive experiments quantify layer relationships and confirm scalability (Aguiar et al., 2022).
• Recursive neural networks: NeuTucker architectures are utilized to model context aggregation in recursive neural models (e.g., Tree-LSTM), mediating between full tensor aggregation and simple summation. The Tucker approximation allows efficient parameter control and increased expressivity for tree-structured data (Castellana et al., 2020).

5. Uniqueness, Identifiability, and Theoretical Properties

NeuTucker solutions are uniquely characterized under specific conditions:

• Essential uniqueness: For nonnegative factors, if each mode’s unfolding admits an essentially unique NMF, or pure-source-dominant conditions are met (e.g., separability in factors or core), the overall decomposition is unique up to permutation and positive diagonal scaling (Zhou et al., 2014).
• Rank selection: Multilinear ranks are pragmatic hyperparameters set by singular value decay, model deviance, or BIC optimality (Li et al., 2013). Tucker models offer flexible, mode-specific rank control not possible in CP decomposition.
• Consistency: Maximum likelihood estimators converge to the best rank-constrained representation in KL sense; exact low-rank Tucker signals yield asymptotic normality for parameter estimates (Li et al., 2013).

6. Algorithmic Considerations and Scalability

Efficient implementation is critical for large-scale tensors:

• Low-rank approximation (LRA): Initial HOSVD or randomized LRA can compress the input tensor, reducing per-iteration gradient cost from $\mathcal{O}(\prod_n I_n R)$ to $\mathcal{O}(N I R^2 + N R^{N+1})$ (Zhou et al., 2014).
• Initialization: Nonnegative HOSVD or random initialization serve practical starting points.
• Optimization: HALS and APG are empirically superior for balancing speed and accuracy; block principal pivoting achieves exact nonnegative least squares for factor updates.
• Scalability: NeuTucker models converge rapidly for moderate tensors and scale to $N=10^4, L=20$ in multilayer graphs, with inherent monotonicity of KL objective and stability under multistart strategies (Aguiar et al., 2022).

7. Empirical Benchmarks and Comparative Analysis

Comprehensive empirical results substantiate NeuTucker’s effectiveness:

Model	MAE	RMSE	R²
NeuTucker (M1)	11.64 ± 2.47	20.11 ± 3.76	0.353 ± 0.195
MLP (M2)	12.32 ± 1.99	21.10 ± 2.97	0.066 ± 0.152
Linear (M3)	11.82 ± 1.15	20.71 ± 2.62	0.177 ± 0.080
Pairwise (M4)	13.25 ± 1.31	22.61 ± 2.03	0.081 ± 0.118
Deep Pairwise(M5)	12.63 ± 2.22	21.34 ± 4.71	0.011 ± 0.103

For Tree-LSTM benchmarks (Castellana et al., 2020), NeuTucker models achieve comparable or superior classification accuracy with dramatically fewer parameters than full tensor or sum-based aggregators. In neuroimaging, Tucker regression yields lower misclassification and richer pattern recovery than CP.

A plausible implication is that NeuTucker decomposition provides a principled and empirically robust framework for modeling nonlinear, High-dimensional tensor-valued data across domains where mode interactions and sparsity are critical.

---

For further technical detail and formal proofs, see (Fan et al., 5 Dec 2025, Li et al., 2013, Aguiar et al., 2022, Zhou et al., 2014), and (Castellana et al., 2020).