Uncertainty-Driven Kernel Tensor Learning (UKTL)

• The paper introduces UKTL, a framework that preserves multi-way tensor structure by comparing mode-specific subspaces with adaptive uncertainty-driven weighting.
• It leverages a tensor encoder with a Higher-order Transformer and Grassmann-inspired subspace kernels to effectively capture nonlinear similarities in complex data.
• The framework employs a Nyström approximation for scalability and demonstrates improved performance in skeletal action recognition benchmarks over traditional methods.

Uncertainty-driven Kernel Tensor Learning (UKTL) is a kernel-learning framework for structured $M$-mode tensor data that compares tensors through mode-wise subspaces derived from tensor unfoldings and adaptively reweights those subspaces according to estimated uncertainty. Introduced for higher-order tensor sequences, especially skeletal action sequences, UKTL combines a tensor encoder, Grassmann-inspired subspace kernels, uncertainty-aware mode weighting, and a Nyström linearization with learned pivots, yielding an end-to-end trainable method that is intended to preserve multi-way structure, model nonlinear similarity, remain scalable, and expose mode-wise interpretability (Wang et al., 20 Mar 2026).

1. Problem formulation and representational setting

UKTL addresses learning from structured higher-order tensor sequences without flattening them into vectors. The motivating application in the original presentation is skeletal action recognition, where a tensor sequence contains interactions across multiple modes such as the feature or channel mode, the hyper-edge or joint-group mode, and the temporal mode. The framework is positioned against four limitations of common alternatives: vectorization destroys structure; linear tensor decomposition preserves structure but cannot model strong nonlinear relations; conventional kernel methods are nonlinear but typically operate on vectorized data and are expensive at scale; and equal treatment of all modes ignores the fact that some modes may be noisy or less discriminative (Wang et al., 20 Mar 2026).

For skeleton data, the raw tensor is written as

$\mathcal{X} \in \mathbb{R}^{d \times J \times T},$

where $d$ is the coordinate or feature dimension, $J$ is the number of joints, and $T$ is the number of frames. After preprocessing and structured encoding, the tensor used by UKTL is written as

$$\mathcal{X} \in \mathbb{R}^{d' \times N_\xi \times \tau},$$

where $d'$ is the learned feature dimension, $N_\xi=\binom{J}{3}$ is the number of 3-joint hyper-edges, and $\tau$ is the number of temporal blocks. This tensorial representation is central: UKTL treats each mode as a structured axis to be modeled explicitly rather than as a dimension to be collapsed (Wang et al., 20 Mar 2026).

The framework’s high-level components are a tensor encoder, mode-wise subspace extraction, a structured tensor kernel, and uncertainty-aware reweighting with Nyström approximation. The resulting design compares tensors as collections of mode-specific subspaces and modulates each subspace by learned confidence, rather than comparing raw arrays directly (Wang et al., 20 Mar 2026).

2. Tensor encoder and higher-order structure preservation

UKTL begins with a lightweight backbone described as an MLP plus a Higher-order Transformer (HoT). The sequence is first split into temporal blocks, and each block is encoded by a per-joint MLP applied to

$\mathbf{X}_t \in \mathbb{R}^{d \times J}.$

The HoT layer is then used to aggregate hyper-edge interactions through the formulation

$\mathcal{X} \in \mathbb{R}^{d \times J \times T},$0

$\mathcal{X} \in \mathbb{R}^{d \times J \times T},$1

$\mathcal{X} \in \mathbb{R}^{d \times J \times T},$2

In the paper’s description, this encoder captures higher-order relations across joints, hyper-edges, and time, and produces the structured tensor on which the kernel stage operates (Wang et al., 20 Mar 2026).

This encoder is not presented as a generic preprocessor detached from the kernel. Rather, it supplies the multi-way object whose mode-wise unfoldings will later be mapped to subspaces. UKTL therefore preserves higher-order structure at both stages: first in the encoder that forms the tensor, and then in the kernel that compares the resulting mode-wise subspaces (Wang et al., 20 Mar 2026).

3. Mode-wise subspaces and structured tensor kernels

For a general $\mathcal{X} \in \mathbb{R}^{d \times J \times T},$3-th order tensor

$\mathcal{X} \in \mathbb{R}^{d \times J \times T},$4

UKTL forms the mode-$\mathcal{X} \in \mathbb{R}^{d \times J \times T},$5 unfolding

$\mathcal{X} \in \mathbb{R}^{d \times J \times T},$6

For the main tensor form used in the paper,

$\mathcal{X} \in \mathbb{R}^{d \times J \times T},$7

the three unfoldings are

$\mathcal{X} \in \mathbb{R}^{d \times J \times T},$8

Each unfolding is subjected to truncated SVD,

$\mathcal{X} \in \mathbb{R}^{d \times J \times T},$9

where $d$0 contains the top-$d$1 left singular vectors. These vectors define a $d$2-dimensional mode-wise subspace

$d$3

with projection matrix

$d$4

The geometric comparison is therefore performed on Grassmannian subspaces rather than on raw tensors (Wang et al., 20 Mar 2026).

The elementary mode-wise similarity uses the projection-kernel quantity

$d$5

and is extended to an RBF-style Grassmann kernel

$d$6

UKTL then composes mode-wise kernels in three ways. The product kernel is

$d$7

The sum kernel is

$d$8

The sum-product kernel interpolates between them:

$d$9

with $J$0 (Wang et al., 20 Mar 2026).

The paper interprets the sum term as robust, mode-wise additive evidence, the product term as strict multi-mode agreement, and the mixture as a balance between robustness and discriminative power. This composition is how UKTL models both independent mode contributions and cross-mode coordination within one kernelized similarity function (Wang et al., 20 Mar 2026).

4. Uncertainty-aware subspace weighting

The principal novelty beyond the underlying KTL formulation is uncertainty-driven weighting of subspace directions. For each tensor mode $J$1, a Multi-mode SigmaNet (MSN) takes the mode-wise projection matrix and predicts an uncertainty vector

$J$2

The paper describes MSN as a small FC network with scaled sigmoid activation, producing positive bounded outputs that are interpreted as mode- and basis-direction-specific confidence scores (Wang et al., 20 Mar 2026).

The basis is then rescaled according to

$J$3

with the division applied row-wise or element-wise across basis directions. All kernel formulas are updated by replacing $J$4 with $J$5. For example, the uncertainty-aware product kernel becomes

$J$6

The stated effect is to suppress unreliable directions so that the kernel compares what is trustworthy rather than merely what is present (Wang et al., 20 Mar 2026).

The appendix supplies a maximum-likelihood rationale for this weighting. Each row of a mode-wise basis is modeled as noisy:

$J$7

The negative log-likelihood becomes, up to constants,

$J$8

leading to the weighted objective

$J$9

Within this derivation, the scaling by $T$0 encodes the principle that higher uncertainty yields lower weight (Wang et al., 20 Mar 2026).

The practical significance stated in the paper is robustness to noise, occlusion, viewpoint variation, and modality imbalance, together with interpretability because the uncertainty vectors indicate which modes and which subspace directions are trusted (Wang et al., 20 Mar 2026).

5. Nyström linearization, training objective, and computational profile

To avoid the cost of dense kernel matrices, UKTL employs a Nyström approximation with differentiable pivot selection. Rather than using fixed pivots, it learns $T$1 pivots $T$2 via soft $T$3-means:

$T$4

where $T$5 is the soft assignment vector for sample $T$6 (Wang et al., 20 Mar 2026).

The kernel blocks are then defined by

$T$7

After eigendecomposing

$T$8

the paper defines

$T$9

and constructs the Nyström embedding

$$\mathcal{X} \in \mathbb{R}^{d' \times N_\xi \times \tau},$$0

where $$\mathcal{X} \in \mathbb{R}^{d' \times N_\xi \times \tau},$$1 is the mean-centered version. This yields a low-dimensional feature map of size $$\mathcal{X} \in \mathbb{R}^{d' \times N_\xi \times \tau},$$2, after which a final FC layer performs classification (Wang et al., 20 Mar 2026).

The full pipeline is described in nine stages: preprocessing of raw sequences; MLP embedding; HoT encoding; mode-wise unfolding; SVD-based subspace extraction; MSN uncertainty estimation; uncertainty-aware kernel computation; Nyström approximation; and a final classifier. The training objective combines classification loss with uncertainty regularization:

$$\mathcal{X} \in \mathbb{R}^{d' \times N_\xi \times \tau},$$3

Here $$\mathcal{X} \in \mathbb{R}^{d' \times N_\xi \times \tau},$$4 is cross-entropy, $$\mathcal{X} \in \mathbb{R}^{d' \times N_\xi \times \tau},$$5 controls uncertainty regularization, and $$\mathcal{X} \in \mathbb{R}^{d' \times N_\xi \times \tau},$$6 denotes all parameters (Wang et al., 20 Mar 2026).

The reported complexity is

$$\mathcal{X} \in \mathbb{R}^{d' \times N_\xi \times \tau},$$7

with inference cost

$$\mathcal{X} \in \mathbb{R}^{d' \times N_\xi \times \tau},$$8

The paper attributes scalability to the shared MLP+HoT backbone, subspace extraction that scales linearly in $$\mathcal{X} \in \mathbb{R}^{d' \times N_\xi \times \tau},$$9, kernel evaluation that scales linearly in $d'$0, and the Nyström approximation that avoids full $d'$1 kernel matrices. It also reports throughput and memory use competitive with strong deep baselines, together with greater parameter efficiency than large transformer models (Wang et al., 20 Mar 2026).

6. Empirical behavior, ablations, and relation to adjacent uncertainty-kernel research

UKTL is evaluated on NTU-60, NTU-120, and Kinetics-Skeleton. The paper reports consistent improvements over KTL, the corresponding kernel tensor learning baseline without uncertainty-aware weighting (Wang et al., 20 Mar 2026).

Benchmark	KTL	UKTL
NTU-60	92.5 / 97.1	93.1 / 97.3
NTU-120	88.8 / 90.3	90.0 / 91.4
Kinetics-Skeleton	38.9 / 61.9	39.2 / 62.3

For multimodal fusion, the reported best results are 95.5 / 98.5 on NTU-60 and 92.8 / 94.0 on NTU-120 when combining skeleton, RGB, and depth. The paper also states that fusion consistently improves performance over skeleton alone or two-modality combinations (Wang et al., 20 Mar 2026).

The ablation results are used to characterize how UKTL behaves. Performance improves as subspace order $d'$2 increases and then saturates, with NTU-60 best around $d'$3 and NTU-120 best around $d'$4. A linear kernel performs worst, a polynomial kernel improves on it, and the sum-product Grassmann kernel performs best. Accuracy improves up to about 150 pivots and plateaus beyond 180. Sum-only composition is weaker, product-only is strong, and sum-product is best. The learned mixture weight settles around 0.41 on NTU-60, 0.47 on NTU-120, and 0.58 on Kinetics-Skeleton; the paper interprets this as indicating that cleaner structured datasets prefer more product-like interaction, whereas noisier datasets benefit from more additive robustness (Wang et al., 20 Mar 2026).

A recurrent misconception would be to treat UKTL as either a standard vectorized kernel method or a purely linear tensor method. The formulation in (Wang et al., 20 Mar 2026) is neither: it preserves tensor structure through mode-wise unfoldings and Grassmann subspaces, and it introduces uncertainty-aware reweighting rather than assuming all mode contributions are equally reliable. Another possible misconception is to read UKTL as a Bayesian weight-space uncertainty method. The paper does not do that; uncertainty is attached to subspace directions through MSN and a weighted kernel, not through posterior inference over network parameters (Wang et al., 20 Mar 2026).

In the broader literature, several neighboring lines of work illuminate what UKTL is and is not. The kernel-field frameworks based on the information potential field (IPF) and quantum information potential field (QIPF) estimate localized uncertainty in Gaussian RKHSs, including under distributional shift, by analyzing densities or model prediction spaces through Schrödinger-inspired operators and Hermite-based uncertainty modes (Singh et al., 2020, Singh et al., 2021). Those methods are kernel-driven and uncertainty-driven, but they do not define UKTL as a tensor-subspace kernel framework. Bayesian Tensor Network Kernel Machines introduce a probabilistic tensor-network kernel model with automatic rank and feature selection via sparsity-inducing priors and mean-field variational inference, yielding predictive uncertainty through a Student’s $d'$5 distribution (Kilic et al., 15 Jul 2025). That line is explicitly tensorial and uncertainty-aware, but its mechanism is Bayesian inference over low-rank tensor-network factors rather than uncertainty-weighted Grassmann subspaces. ScaLLA, a scalable approximate linearized Laplace method, learns a compact surrogate kernel for LLA and shows that deliberately biased learned kernels can improve OOD behavior, which suggests a broader principle that uncertainty objectives can justify kernels that are not merely faithful approximations of a reference kernel (Ortega et al., 29 Jan 2026). In that context, UKTL can be situated as a method where the uncertainty objective is built directly into mode-wise tensor comparison rather than added post hoc to a fixed kernel family.

Taken together, these comparisons indicate that UKTL occupies a specific niche: a scalable, end-to-end trainable kernel framework for tensor sequences in which uncertainty enters through confidence-weighted subspace geometry, rather than through Bayesian marginalization in weight space or purely density-based analysis in output space (Wang et al., 20 Mar 2026).