Kronecker Tensor Product

• Kronecker tensor product is an algebraic operation that generalizes the matrix Kronecker product to higher-order tensors, preserving bilinearity, associativity, and spectral properties.
• It enables the systematic construction of large, structured tensors from smaller components, facilitating tensor decompositions, low-rank approximations, and efficient computations.
• Its applications span multilinear algebra, signal processing, data compression, and quantum physics, where it underpins tensor spectral analysis and hypergraph studies.

The Kronecker tensor product is an algebraic operation that generalizes the classical Kronecker product of matrices to higher-order tensors. This operation provides a systematic construction for forming large, highly structured tensors from smaller ones, preserving and extending key properties of bilinearity, associativity, and spectral behavior. The Kronecker tensor product is foundational in multilinear algebra, tensor decompositions, high-dimensional data analysis, signal processing, numerical multilinear algebra, and representation theory.

1. Formal Definition

For matrices $A \in \mathbb{R}^{m \times n}$ and $B \in \mathbb{R}^{p \times q}$, their Kronecker product is the $mp \times nq$ block matrix: $$A \otimes B = \begin{pmatrix} a_{11} B & \cdots & a_{1n} B \ \vdots & \ddots & \vdots \ a_{m1} B & \cdots & a_{mn} B \end{pmatrix}$$ Equivalently, $$(A \otimes B)_{ip + k, jq + \ell} = A_{ij} B_{k\ell}$$ for $0 \leq i < m$, $0 \leq j < n$, $0 \leq k < p$, $0 \leq \ell < q$ (Yoshikawa et al., 2020, 0907.0796).

For order-$N$ tensors $A \in \mathbb{R}^{I_1 \times \cdots \times I_N}$ and $B \in \mathbb{R}^{J_1 \times \cdots \times J_N}$, their Kronecker product $A \otimes B$ is the $(I_1 J_1) \times \cdots \times (I_N J_N)$ tensor given by

$$(A \otimes B)_{(i_1, j_1), \dots, (i_N, j_N)} = A_{i_1, \dots, i_N} B_{j_1, \dots, j_N}$$

This definition is fully consistent with block-tensor partitioning and allows for index grouping and reshaping as required by specific applications (Pickard et al., 2023, Colley et al., 2020, Batselier et al., 2015, Shao, 2012).

2. Algebraic and Structural Properties

The Kronecker tensor product preserves foundational properties of the matrix Kronecker product:

• Bilinearity: $$(\alpha A_1 + \beta A_2) \otimes B = \alpha (A_1 \otimes B) + \beta (A_2 \otimes B)$$.
• Associativity: $(A \otimes B) \otimes C = A \otimes (B \otimes C)$, with suitable interpretation of mode grouping.
• Mixed-Product Property: For suitably dimensioned matrices or tensors, $(A \otimes B)(C \otimes D) = (A C) \otimes (B D)$.
• Trace and Determinant: For matrices, $$\mathrm{tr}(A \otimes B) = \mathrm{tr}(A) \mathrm{tr}(B)$$ and $\det(A \otimes B) = (\det A)^m (\det B)^n$ (Rosas-Ortiz et al., 2013).

For tensors, mode-wise operations interact naturally, e.g., for the mode-$p$ multiplication: $$(A \otimes B) \times_p (X \otimes Y) = (A \times_p X) \otimes (B \times_p Y)$$ Inner products and norms are separable: $$\langle A \otimes B, X \otimes Y \rangle = \langle A, X \rangle \langle B, Y \rangle$$

$\| A \otimes B \|_F = \|A\|_F \|B\|_F$

(Pickard et al., 2023, Colley et al., 2020, 0907.0796).

3. Spectral and Decomposition-Theoretic Remarks

The Kronecker tensor product has a fundamental role in tensor spectral theory:

• If $(\lambda_A, u)$ and $(\lambda_B, v)$ are Z-eigenpairs of symmetric tensors $A$ and $B$, then $(\lambda_A \lambda_B, u \otimes v)$ is a Z-eigenpair of $A \otimes B$ (Colley et al., 2020, Pickard et al., 2023, Shao, 2012).
• Analogous results hold for H-eigenpairs, M-eigentriples, and related spectral generalizations.
• The Kronecker product is multiplicative for both spectral and nuclear tensor norms:
  • $$\| S \otimes T \|_\sigma = \|S\|_\sigma \|T\|_\sigma$$, $\|S \otimes T \|_* = \|S\|_* \|T\|_*$ (Cochrane, 2020).
• These properties support direct analysis of composite systems by analysis of their factors.

In tensor decomposition:

• The Kronecker product of Tucker structures is itself a Tucker structure whose core and factor matrices are Kronecker products of the factors.
• Tensor train decompositions and canonical polyadic decompositions are compatible: the Kronecker product of tensors with structured factorization yields a structured factorization of the product tensor (Pickard et al., 2023, Batselier et al., 2015).

4. Computational Methods and Algorithms

Practical algorithms for decomposing tensors into Kronecker products focus on exploiting separability and low-rank structure:

• TKPSVD: Decomposes a $k$-way tensor as a sum of Kronecker products of $d$ smaller $k$-way tensors. The algorithm involves reshaping, permuting, and performing orthogonal polyadic decompositions. Structural properties such as symmetry, centrosymmetry, and Toeplitz structure are preserved in the Kronecker factors (Batselier et al., 2015).
• Generalized Kronecker Product Decomposition (GKPD): Employs a rearrangement of the target tensor followed by truncated SVD for efficient low-rank approximation—a key technique in deep neural network compression. Compression efficiency grows rapidly with tensor size, e.g., ~26× parameter reduction for a 4-D convolutional kernel with negligible accuracy loss (Hameed et al., 2021).
• Power-Method Schemes: Dominant eigenvectors of a Kronecker-structured tensor are efficiently approximated by power methods that operate on low-rank matrix factors, leveraging the contraction properties of Kronecker products (Colley et al., 2020).

Optimal implementation can be guided by Mathematics of Arrays (MoA) and psi-Calculus, which provide shape- and layout-aware indexing frameworks for high-performance, cache-optimal, deterministic computation of Kronecker products (0907.0796).

5. Applications and Implications

The Kronecker tensor product supports broad applications across mathematical sciences, signal processing, data analysis, and representation theory:

• Multilinear Common Component Analysis (MCCA): Constructs a global covariance tensor from mode-wise covariances via the Kronecker product, enabling model decoupling and mode-wise optimization. MCCA outperforms vectorized and multilinear PCA/CCA in large-scale tensor compression tasks (Yoshikawa et al., 2020).
• Large-Scale Numerical Multilinear Algebra: Stability, computational guarantees, and error propagation in Kronecker-structured Lyapunov and PDE solvers benefit directly from norm multiplicativity (Cochrane, 2020).
• Graph and Hypergraph Problems: Kronecker products of graph adjacency tensors yield rigorous descriptions of product hypergraph structures, spectral properties, and network alignment algorithms (e.g., TAME, $\Lambda$TAME) (Pickard et al., 2023, Colley et al., 2020, Shao, 2012).
• Quantum Physics and Group Representations: The Kronecker product, expressed in the Hubbard operator formalism, is central to the addition of angular momenta and Clebsch–Gordan decomposition in $SU(2) \times SU(2)$ representations. It provides explicit, sparse-combinatorial formulas for key quantities (Rosas-Ortiz et al., 2013).
• Representation Theory: In the modular and classical setting, Krause’s internal tensor product of strict polynomial functors corresponds under the Schur functor to the Kronecker tensor product of symmetric-group representations, with implications for the Kronecker problem and combinatorics of Specht modules (Kulkarni et al., 2015).

6. Connections to Hypergraphs, Dynamics, and Structural Inheritance

Kronecker products of hypergraph adjacency tensors define "Kronecker hypergraphs" whose vertices and edges are Cartesian products of the original hypergraphs. This construction preserves eigenvector centrality, degree sequences, and supports analysis of polynomial dynamics on such products. Notably, the system dynamics of Kronecker-hypergraph–coupled systems can exhibit instability even when constituent systems are stable, due to eigenvalue multiplication (Pickard et al., 2023, Shao, 2012).

Structural properties—such as symmetry or Toeplitz structure—are generically inherited from parent tensors to Kronecker factors within proper decompositions, provided that associated singular values are simple. This enables symmetry-aware tensor factorization and efficient encoding of algebraic invariants (Batselier et al., 2015).

7. Summary Table: Central Algebraic and Spectral Properties

Property Class	Matrix Kronecker Product	Kronecker Tensor Product
Bilinearity/Associativity	Yes	Yes
Mixed-Product Rule	$(A \otimes B)(C \otimes D) = (AC) \otimes (BD)$	Mode-wise extensions hold
Trace/Determinant	$$\operatorname{tr}(A\otimes B) = \operatorname{tr}(A)\operatorname{tr}(B)$$	Per-mode product applicable
Spectral Norm	Multiplicative	Multiplicative
Nuclear Norm	Multiplicative	Multiplicative
Decomposition Compatibility	SVD, CP, Tucker, TT	Tensor analogues via Kronecker

This table summarizes the extension of core matrix product properties to multiway arrays via the tensor Kronecker product (Yoshikawa et al., 2020, Pickard et al., 2023, Cochrane, 2020, Batselier et al., 2015, 0907.0796).

• "Multilinear Common Component Analysis via Kronecker Product Representation" (Yoshikawa et al., 2020)
• "Dominant Z-Eigenpairs of Tensor Kronecker Products are Decoupled and Applications to Higher-Order Graph Matching" (Colley et al., 2020)
• "Kronecker Product of Tensors and Hypergraphs: Structure and Dynamics" (Pickard et al., 2023)
• "Kronecker product in terms of Hubbard operators and the Clebsch-Gordan decomposition of SU(2)xSU(2)" (Rosas-Ortiz et al., 2013)
• "Tensors and n-d Arrays:A Mathematics of Arrays (MoA), psi-Calculus and the Composition of Tensor and Array Operations" (0907.0796)
• "Nuclear Norm Under Tensor Kronecker Products" (Cochrane, 2020)
• "A constructive arbitrary-degree Kronecker product decomposition of tensors" (Batselier et al., 2015)
• "A general product of tensors with applications" (Shao, 2012)
• "Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition" (Hameed et al., 2021)
• "Relating tensor structures on representations of general linear and symmetric groups" (Kulkarni et al., 2015)