Generalized NS-KTR Regression

• The paper introduces a low-rank CP decomposition with nonnegativity and mode-specific hybrid regularization to efficiently capture structural heterogeneity.
• It integrates LASSO, total variation, and ridge penalties, enhancing feature selection, smoothness, and stability in tensor regression.
• The study details a block-alternating ADMM solver with strong theoretical guarantees that ensure identifiability and robust performance in applications like hyperspectral imaging.

Generalized Nonnegative Structured Kruskal Tensor Regression (NS-KTR) is a tensor regression framework that parameterizes the regression coefficient tensor using a low-rank CANDECOMP/PARAFAC (CP) decomposition with nonnegativity and mode-specific regularization. By assigning hybrid penalties (LASSO, total variation, and ridge) tailored to each tensor mode, NS-KTR efficiently models the structural heterogeneity found in multidimensional data, ensuring interpretability and superior predictive accuracy, especially in applications like hyperspectral image analysis and signal processing. NS-KTR supports both linear and logistic regression losses, and employs a block-alternating ADMM solver for scalable and robust estimation.

1. Model Construction and Parameterization

NS-KTR expresses the regression coefficient tensor $$\mathcal{B} \in \mathbb{R}^{I_1 \times \cdots \times I_D}$$ as a sum of $R$ nonnegative rank-1 components via a CP/Kruskal decomposition:

$$\mathcal{B} = \sum_{r=1}^R b_1(:,r) \circ b_2(:,r) \circ \cdots \circ b_D(:,r)$$

where each $B_d \in \mathbb{R}^{I_d \times R}$ is a factor matrix for mode $d$, with columns $b_d(:,r) \geq 0$ for all $r$ (nonnegativity). This formulation dramatically reduces the parameter space from $\prod_{d=1}^D I_d$ to $R \sum_{d=1}^D I_d$, fostering parsimony and interpretability (Wang et al., 24 Sep 2025). The decomposition guarantees the existence of optimal nonnegative approximations under any continuous norm or even Bregman divergence, and ensures model well-posedness (0903.4530).

Coefficient estimation in NS-KTR is posed as the minimization:

$$\min_{\{B_1,\dots,B_D\}} \; f(B_1,...,B_D) + \sum_{d=1}^D h_d(\operatorname{vec}(B_d))$$

where $f(\cdot)$ is a data fidelity term (linear or logistic regression loss) and $h_d(\cdot)$ is the mode-specific regularization.

2. Mode-Specific Hybrid Regularization

NS-KTR introduces flexible, mode-wise regularization to address the structural heterogeneity across tensor modes:

• LASSO penalty: $\lambda_{d1} \|\beta_d\|_1$ promotes sparsity, leading to feature selection in mode $d$.
• Total Variation/Fused penalty: $\lambda_{d2} \|D_d \beta_d\|_1$, with $D_d$ as a difference operator, encourages piecewise constant or regular variation, ideal for spatial or segmentation-sensitive modes.
• Ridge penalty: $\frac{\lambda_{d3}}{2} \|\beta_d\|_2^2$ stabilizes the estimation in noisy settings.
• Nonnegativity indicator: $\iota_{\ge 0}(\beta_d)$ restricts factor entries to be nonnegative.

The general regularizer for mode $d$ is:

$$h_d(\beta_d) = \lambda_{d1} \|\beta_d\|_1 + \lambda_{d2} \|D_d\beta_d\|_1 + \frac{\lambda_{d3}}{2} \|\beta_d\|_2^2 + \iota_{\ge 0}(\beta_d)$$

This structure allows NS-KTR to tailor penalties to data properties, e.g., smooth spectral bands versus spatial discontinuities (Wang et al., 24 Sep 2025).

3. Existence, Uniqueness, and Identifiability

Nonnegativity constraints, as shown by (0903.4530) and (Qi et al., 2014), ensure NS-KTR admits optimal solutions under arbitrary norms or Bregman divergences for any nonnegative tensor. Uniqueness of best nonnegative rank-$r$ approximations holds generically, with failure sets forming algebraic hypersurfaces negligible in measure. For rank-1, any optimal approximation is nonnegative by construction. Deflation is provably invalid for positive tensors: joint rank-$r$ approximation is required (Qi et al., 2014).

Identifiability results from semialgebraic geometry (Qi et al., 2015) indicate that for nonnegative rank $r$ strictly below the generic rank, the decomposition (unless on a cell boundary of $D_r$) is unique. For $r \leq 3$, uniqueness always holds provided the mode dimensions are large enough.

4. Optimization Methodology

NS-KTR employs a block-alternating optimization scheme using ADMM, breaking the nonconvex problem into mode-wise convex subproblems:

1. Alternating update: Each $B_d$ is updated by solving:

$$B_d^{t+1} = \arg\min_{\beta_d} \, g(\beta_d; \text{fixed factors}) + h_d(\beta_d)$$

1. ADMM splitting: The objective is split by introducing auxiliary variables for LASSO ($z_1$) and TV ($z_2$) terms, yielding subproblems such as:

$$\min_{x,z_1,z_2} \, g(x) + \lambda_1 \|z_1\|_1 + \lambda_2 \|z_2\|_1 + \frac{\lambda_3}{2}\|x\|_2^2 \quad\text{ s.t. } x=z_2,\, x \ge 0$$

The augmented Lagrangian is minimized alternately: - Primal updates (projected or Newton-type) - Proximal updates for LASSO/TV - Dual variable updates

For linear regression $g(x)$, closed-form solutions can be found; for logistic regression, Newton-based updates with line search are used.

5. Theoretical and Empirical Properties

NS-KTR leverages strong theoretical guarantees:

• Well-posedness: Existence of solutions for all norm and divergence choices (0903.4530).
• Uniqueness and identifiability: Generic uniqueness ensures interpretability and model stability (Qi et al., 2014).
• Statistical risk bounds and generalization: Tensor-based regularizers (overlapped/latent/scaled latent norms) control excess risk and are provably superior to vector/matrix-based methods (Wimalawarne et al., 2015).
• Algorithmic performance: Nonconvex projected gradient descent yields favorable error rates with practical scalability when projections are computable (Chen et al., 2016).

Empirical results on synthetic signals (e.g., Gradient, Floor, Wave, Fading Cross) and real hyperspectral data demonstrate superiority of NS-KTR with appropriate regularization, yielding lower estimation errors and improved generalization compared to baseline tensor regression approaches (Wang et al., 24 Sep 2025).

6. Handling Structured Constraints and Kruskal Rank

Efficient verification of Kruskal rank is vital for NS-KTR identifiability. Recent randomized hashing and dynamic programming algorithms provide high-probability and deterministic guarantees for Kruskal rank verification across binary, finite, and integer settings, with practical runtime scaling as $\mathcal{O}(dk (nM)^{\lceil k/2 \rceil})$. These methods can be adapted to NS-KTR, especially leveraging bounded coefficient spaces from nonnegativity constraints, to ensure identifiability of learned factors (Zhou, 6 Mar 2025).

Advanced approaches, such as those using Riemannian optimization on manifolds, allow integration of nonnegativity and other structure directly into the tensor factorization, simplifying optimization with certificates via duality gap expressions. These frameworks are amenable to NS-KTR's requirements for imposing mode-specific structure while maintaining low rank (Naram et al., 2023).

7. Applications and Structural Heterogeneity

NS-KTR's ability to handle structural heterogeneity is central to its success:

• Hyperspectral imaging: Spectral modes benefit from smoothness, spatial modes from piecewise constancy; NS-KTR assigns distinct regularization for each.
• Neuroimaging: Supports nonnegative activation modeling and anatomical regularity.
• Chemometrics/remote sensing/material science: Nonnegativity ties components to physical phenomena such as absorbance or reflectance.
• Financial tensors/social networks: Regularization can be matched to temporal, asset, or network-specific structure.

Real-world experiments show enhanced regression and classification accuracy, with regularized nonnegativity yielding stability and interpretability in recovered factors (Wang et al., 24 Sep 2025).

8. Limitations and Future Directions

Limitations include sensitivity to regularization parameter choices and potential challenges in optimizing projections for highly structured constraint sets (especially combining rank, sparsity, and nonnegativity simultaneously). As the theoretical foundation for structured priors (such as deep generative models for tensor factors) evolves, phase transitions in information recovery and computational hardness may emerge as practical limitations (Luneau et al., 2020). Efficient algorithm design and adaptive regularization strategies for high-dimensional settings remain important ongoing research areas.

Concluding Remarks

Generalized Nonnegative Structured Kruskal Tensor Regression synthesizes advances in nonnegative tensor decompositions, mode-specific regularization, identifiability theory, and algorithmic scalability. It achieves interpretable, accurate, and physically plausible regression for complex multidimensional data, with established theoretical underpinnings and robust empirical validation in signal processing and hyperspectral analysis. The framework is extensible to broader domains where nonnegativity and structural constraints are foundational.