Tensor LDA: Multilinear Discriminant Analysis

• Tensor LDA is a supervised subspace learning method that extends classical LDA to tensor data, preserving its multi-dimensional structure while mitigating the curse of dimensionality.
• It employs mode-specific projection matrices to maximize between-class variance and minimize within-class variance, ensuring robust discrimination.
• Tensor LDA has practical applications in imaging, video action recognition, medical neuroimaging, and time-series prediction, offering computational efficiency and enhanced interpretability.

Tensor-Based Discriminant Analysis (Tensor LDA), also broadly known as Multilinear Discriminant Analysis (MDA), comprises a family of supervised subspace learning methods that generalize classical Linear Discriminant Analysis from vectorial to high-order (tensor) data. Such techniques directly exploit the multilinear structure of signals naturally represented as multi-way arrays, including images, videos, hyperspectral cubes, multichannel time series, and neuroimaging data. The core objective is to find mode-specific projection matrices—or more general low-rank tensor projectors—that maximize between-class variance while minimizing within-class variance in the projected tensor domain. This approach preserves spatial, temporal, or other intrinsic structural dependencies typically lost during data vectorization, while mitigating the curse of dimensionality and the small-sample problem encountered in conventional LDA. Over the last decade, Tensor LDA methods have advanced significantly with innovations in model criteria (multiclass and class-specific), tensor decompositions (Tucker, CP, tensor-train, block-term), optimization schemes (alternating mode-wise updates, transform-domain reductions, convex regularization), and applications in domains such as facial recognition, action recognition, time-series prediction, medical imaging, and brain-computer interfacing.

1. Mathematical Framework: From LDA to Tensor LDA

Classical LDA seeks a linear mapping $W$ that projects vectorized data $x \in \mathbb{R}^I$ onto a subspace where the trace-ratio $$J(W) = \operatorname{tr}(W^\top S_B W) / \operatorname{tr}(W^\top S_W W)$$ is maximized, with $S_B$ and $S_W$ the between- and within-class scatter matrices. For tensor data $$\mathcal{X} \in \mathbb{R}^{I_1 \times \cdots \times I_K}$$, direct vectorization leads to statistical, computational, and structural deficiencies.

Tensor LDA directly models scatter in the tensor domain. The approach replaces vector-matrix products with mode-wise multilinear projections or tensor–tensor products. The most common formulation seeks a collection of mode-specific projection matrices $$\{ U^{(k)} \in \mathbb{R}^{I_k \times R_k} \}_{k=1}^K$$, yielding projected tensors: $$\mathcal{Y}_i = \mathcal{X}_i \times_1 U^{(1)\top} \times_2 \cdots \times_K U^{(K)\top},$$ where $\times_k$ denotes the $k$-mode product. Within-class scatter $\mathcal{S}_W$ and between-class scatter $\mathcal{S}_B$ are defined analogously, either through mode-wise unfoldings and Kronecker contraction (Kong et al., 2012, Zeng et al., 2014), or via fully tensorial generalizations such as t-products, L-products, or transform-domain products (Ozdemir et al., 2022, Dufrenois et al., 2022, Bouallala et al., 2024).

The Fisher criterion is extended to: $$J(\{ U^{(k)} \}) = \frac{\sum_c N_c \| \bar{\mathcal{Y}}_c - \bar{\mathcal{Y}} \|_F^2} {\sum_c \sum_{i: y_i=c} \| \mathcal{Y}_i - \bar{\mathcal{Y}}_c \|_F^2},$$ or, mode-wise,

$$J^{(k)}(U^{(k)}) = \frac{ \operatorname{tr} \left( U^{(k)\top} S_B^{(k)} U^{(k)} \right) }{ \operatorname{tr} \left( U^{(k)\top} S_W^{(k)} U^{(k)} \right) }.$$

This framework subsumes both global multiclass discriminant analysis and, with appropriate objective reformulation, class-specific discrimination in a one-vs-rest paradigm (Tran et al., 2017).

2. Key Model Variants and Tensor Structures

Tensor LDA covers a spectrum of models varying in the form of multilinear projections and criteria:

• Tucker-based Multilinear Discriminant Analysis (MDA): Projects via mode-wise matrices. Optimization is performed by alternating maximization of mode-$k$ Rayleigh quotients or trace-ratios, often using Kronecker contractions and mode-$k$ scatter analysis (Kong et al., 2012, Zeng et al., 2014). The method is widely adopted for moderate-order tensors.
• Class-Specific Multilinear Discriminant Analysis (MCSDA): Employs a one-vs-rest criterion, learning mode-wise projectors that minimize the spread of positive samples around their class mean while maximizing the separation from negatives. The alternating algorithm updates each projector by solving $S_O^k v = \lambda S_I^k v$, with out-of-class ($S_O^k$) and in-class ($S_I^k$) scatter matrices defined after partial projection (Tran et al., 2017).
• Tensor-Train Discriminant Analysis (TTDA): Assumes the discriminant subspace admits a tensor-train factorization, yielding dramatic parameter compression. Multi-branch TT architectures (2-way, 3-way) further reduce the computational burden by splitting the optimization over branches (Sofuoglu et al., 2019).
• Transform-Domain Tensor LDA: Generalizes the criterion to the space of tensors under t-products or other invertible transform-induced multiplications. The scatter tensors are constructed in the frequency domain, and the discriminant subspace becomes a set of block-diagonal matrices, decoupling the learning into per-slice (e.g., per-frame) eigenvalue problems (Ozdemir et al., 2022, Dufrenois et al., 2022, Bouallala et al., 2024).
• Low-Rank Discriminant Structure (Tucker, CP): Imposes low CP or Tucker rank directly on the discriminant tensor, leading to further gains in sample efficiency and interpretability, with provable minimax-optimal estimation and risk bounds (Chen et al., 2024, Chen et al., 2024).
• Block-Term Tensor Discriminant Analysis (BTTDA): Extracts multiple independent Tucker-structured blocks, enhancing flexibility and ability to capture independent discriminant processes, especially for neuroimaging and EEG/ERP data (Kerchove et al., 6 Nov 2025).
• Generalized Difference Subspace (n-mode GDS): Constructs per-mode discriminative “difference” subspaces using eigenanalysis of class-wise average projectors, then integrates multiple modes on a Grassmann product manifold with weighted geodesic distances, optimized by Fisher score weighting (Gatto et al., 2019).

3. Optimization Algorithms and Implementation

The dominant optimization paradigm is alternating mode-wise maximization: for each mode, holding other projectors fixed, one updates the current mode via a generalized eigenproblem or trace-ratio maximization: $S_B^{(k)} v = \lambda S_W^{(k)} v,$ with $S_B^{(k)}$, $S_W^{(k)}$ expressing scatter in mode-$k$ after contraction along the other modes (Kong et al., 2012, Zeng et al., 2014, Tran et al., 2017). Convergence is typically fast (fewer than 30 iterations suffices (Kong et al., 2012)). Singular or ill-conditioned scatter matrices are regularized by re-estimation of small eigenvalues in the transform domain or by adding small positive diagonal corrections (Ozdemir et al., 2022).

When the scatter tensors are defined via t-products or L-products, the entire optimization decomposes into independent per-slice problems, typically solved by Newton–Lanczos, QR iterations, or generalized eigenvalue solvers (Ozdemir et al., 2022, Dufrenois et al., 2022, Bouallala et al., 2024).

For models imposing low-rank structure (Tucker, CP, block-term), tensor power iterations, higher-order orthogonal iterations (HOOI), or alternating least-squares (ALS) are used; in the CP case, iterative projections with randomized composite PCA initialization guarantee global convergence under weak conditions (Chen et al., 2024, Chen et al., 2024).

Sparsity-regularized Tensor LDA formulations, as used in TULIP’s CATCH, apply block-coordinate descent and group LASSO penalties to exploit element-level sparsity, supported by efficient Fortran-level routines and Kronecker-structured covariance estimation (Pan et al., 2019).

4. Comparison with Classical and Alternative Methods

Tensor LDA preserves multiway structure, drastically reducing parameter and computational complexity compared to vectorization-based LDA. For a $K$-mode tensor with dimensions $(I_1,...,I_K)$, the parameter count for Tucker-based models is $\sum_{k=1}^K I_kR_k$, compared to $\prod_{k=1}^K I_k R_k$ in the fully vectorized case (Tran et al., 2017).

Vector-based Class-Specific Discriminant Analysis (CSDA) and LDA, while theoretically optimal for Gaussian models, are impractical for high-order data due to the curse of dimensionality and collapse the internal structure (spatial correlations, channel dependencies). Tensor-Train and multi-branch TTDA outperform Tucker-MDA in scenarios where data are very high-order and storage is limiting: multi-branch TTDA achieves similar or superior recognition accuracy with orders-of-magnitude less storage and runtime (Sofuoglu et al., 2019).

Block-term approaches such as BTTDA offer a flexible compromise between Tucker (single block) and PARAFAC (sum of rank-1 tensors), capturing multiple independent discriminant structures and improving interpretability (Kerchove et al., 6 Nov 2025).

Regularized tensor LDA approaches (CATCH) explicitly address high-dimensionality with limited sample sizes and covariate adjustment, outperforming matrix LDA, vector-ℓ1 classifiers and sparse SVMs on sMRI and multiclass sensor datasets (Pan et al., 2019).

5. Empirical Performance and Applications

Tensor LDA methods consistently outperform their vector-based and non-multilinear alternatives across diverse domains:

• Imaging: On ORL face verification, MCSDA attains 95.7% mAP, 5–10 points higher than MDA, with vastly reduced training time compared to vector CSDA (Tran et al., 2017). On large face/action datasets (FEI, RPPDI, UTD-MHAD), RHOMLDA (DCT domain) yields up to 97.45% mean accuracy, exceeding Tucker-based CMDA/DGTDA (Ozdemir et al., 2022).
• Video Action Recognition: MLDANet achieves 78.93% accuracy on UCF11, outperforming both PCANet and classical tensor-object pipelines, with convolutional architectures facilitating the direct learning of multilinear filters (Zeng et al., 2014).
• Time-Series and Financial Data: MCSDA achieves the best F1-score (46.7%) on stock mid-price movement prediction with tensor-encoded limit order book data (Tran et al., 2017).
• Medical Imaging and Neuroimaging: CATCH consistently yields the lowest error rates on 3D sMRI (22.79% binary, 35.22% multiclass) compared to all matrix or vector-ℓ1 methods, functioning efficiently in high dimensions with penalty tuning and covariate correction (Pan et al., 2019).
• EEG/ERP and BCI: BTTDA delivers highest ROC-AUC (91.25%) and accuracy (64.52%) for ERP/MI decoding among HODA-class methods, with block-term models supporting more robust and interpretable feature extraction than single-block approaches (Kerchove et al., 6 Nov 2025).
• Manifold and Subspace Learning: n-mode GDS with weighted geodesic metrics provides state-of-the-art gesture and action recognition without deep pretraining, matching or exceeding CNN performance on UCF-101/HMDB-51 (Gatto et al., 2019).

Robustness to missing data and finite-sample effects is now supported theoretically and empirically in modern high-dimensional variants, with minimax error bounds established for low-rank Tucker and CP discriminant models (Chen et al., 2024, Chen et al., 2024).

6. Extensions, Implications, and Recent Trends

Modern developments in Tensor LDA include:

• Transform-Domain and Manifold Extensions: The use of t-products and transform-domain algebra allows natural and direct solution of multilinear trace-ratio problems, leading to efficient algorithms and improved stability, as each frontal slice corresponds to a classical eigenproblem (Ozdemir et al., 2022, Dufrenois et al., 2022, Bouallala et al., 2024).
• Scalable Low-Rank and Sparse Models: Tucker and CP low-rank discrimination, with warm-start algorithms and randomized projections, allow handling of ultra-high dimensional spaces (20⁴, 30³), supporting optimal statistical rates and robust convergence (Chen et al., 2024, Chen et al., 2024).
• Block-Term and Multi-Branch Generalizations: Block-term BTTDA generalizes HODA/PARAFAC, optimizing flexibility/expressiveness with sequential block extraction; multi-branch TTDA achieves comparable accuracy to Tucker-based methods at 100× lower memory and computation (Kerchove et al., 6 Nov 2025, Sofuoglu et al., 2019).
• Software Frameworks: Dedicated toolboxes such as TULIP implement sparse, penalized, and covariate-adjusted versions of tensor LDA for practical use in high-dimensional multitask classification (Pan et al., 2019).

A plausible implication is that further integration of robust tensor LDA schemes with deep networks, online learning, and adaptive feature pooling can yield powerful yet interpretable tools for high-order structured data across scientific domains.

7. Summary Table: Representative Tensor LDA Models

Model	Discriminant Structure	Optimization
Tucker-MDA	Mode-wise matrices (Tucker)	Alternating eigenproblems
TTDA, 2WTTDA, 3WTTDA	Tensor-train (TT) factorization	Alternating core updates
MCSDA	One-vs-rest modewise	Alternating class-specific
Transform-domain (t,c-prod.)	Per-frontal-slice projections	Blockwise trace/eigenratio
BTTDA	Multiple Tucker blocks	Deflation + alternating ALS
CATCH (TULIP)	Group-sparse tensors	Block-coord. / $\ell_1$

These models collectively enable the extraction of discriminant subspaces that preserve and exploit the inherent structure of tensor data, achieving statistically efficient, computationally tractable, and empirically robust classification across a wide array of modalities (Tran et al., 2017, Kong et al., 2012, Ozdemir et al., 2022, Kerchove et al., 6 Nov 2025, Sofuoglu et al., 2019, Pan et al., 2019, Zeng et al., 2014, Bouallala et al., 2024, Chen et al., 2024, Chen et al., 2024, Gatto et al., 2019, Dufrenois et al., 2022).