--- title: 'Kronecker Delta Rule: Definitions' url: https://www.emergentmind.com/topics/kronecker-delta-rule type: topic --- # Kronecker Delta Rule: Definitions The expression **Kronecker Delta Rule** denotes several mathematically distinct constructions. In tensor analysis, it is the rule expressing totally symmetric isotropic tensors as sums of products of Kronecker deltas. In multiway statistics, “Kronecker delta covariance structure” denotes a separable covariance model constructed from Kronecker products of mode-specific covariance factors. In algebraic combinatorics, a “Kronecker rule” concerns Kronecker coefficients of symmetric-group representations and is unrelated to the delta symbol. In recurrent linear attention, the Kronecker Delta Rule is an associative-memory update that applies the ordinary Delta Rule to a Kronecker-product feature. These usages share terminology but not a common theorem. ## 1. Terminological distinctions and elementary identities The ordinary Kronecker delta is the equality indicator \[ \delta_{ij}= \begin{cases} 1,&i=j,\\ 0,&i\ne j. \end{cases} \] It is the entry of the identity matrix \[ I_m=(\delta_{ij})_{i,j=1}^m. \] Its fundamental contraction identities include \[ \delta_{ij}V^j=V^i,\qquad \delta_{ij}\delta_{jk}=\delta_{ik},\qquad \delta_{ii}=n. \] Under Einstein summation, $\delta_{ij}V^j$ selects the component with index $i$, while a closed contraction such as $\delta_{ii}$ produces the dimension $n$. The ordinary Kronecker product, denoted by $\otimes$, is a matrix construction rather than an equality indicator. The array-statistical papers also use \[ A\otimes^i B=B\otimes A, \] calling $\otimes^i$ an “inverse Kronecker product.” The superscript $i$ is a reversal of ordering associated with array-vectorization conventions; it does not denote matrix inversion. Consequently: - $\delta_{ij}$ is a scalar equality indicator; - $I=(\delta_{ij})$ is an identity matrix; - $A\otimes B$ is a block or multiway matrix product; - $A\otimes^i B$ is the reversed-order Kronecker product used in the array notation. The phrase “Kronecker Delta Rule” is therefore context-dependent. The tensor-integral literature uses it for a delta-product expansion, whereas the multiway covariance literature uses “Kronecker delta covariance structure” for a separable covariance model, and the attention literature uses it for an update rule on vectorized matrix memories. ## 2. Delta products for isotropic tensors For a totally symmetric isotropic tensor in an $n$-dimensional Euclidean space, all odd ranks vanish: \[ \widetilde I^{i_1\cdots i_{2k+1}}_{(2k+1)}=0. \] For even rank $2k$, rotational invariance and total symmetry require every index to be paired with another index. The number of distinct pairings is \[ (2k-1)!!=\frac{(2k)!}{(2!)^k k!}. \] The Kronecker Delta Rule for rank $2k$ is \[ \widetilde I^{i_1\cdots i_{2k}}_{(2k)} = \frac{1}{n(n+2)\cdots(n+2k-2)} \sum_{\text{distinct pairings}} \delta_{i_{a_1}i_{b_1}}\cdots \delta_{i_{a_k}i_{b_k}}. \] Equivalently, using a sum over all permutations, \[ \widetilde I^{i_1\cdots i_{2k}}_{(2k)} = \frac{1}{n(n+2)\cdots(n+2k-2)} \frac{1}{(2k)!!} \sum_{\sigma} \delta_{\sigma(i_1)\sigma(i_2)} \cdots \delta_{\sigma(i_{2k-1})\sigma(i_{2k})}. \] The normalization is fixed by \[ \widetilde I^{i_1i_1\,i_2i_2\,\cdots\,i_ki_k}_{(2k)}=1. \] The recurrence form is \[ \widetilde I^{i_1i_2\cdots i_{2k}}_{(2k)} = \frac{1}{n+2k-2} \sum_{a=2}^{2k} \delta_{i_1i_a} \widetilde I^{i_2\cdots \widehat{i_a}\cdots i_{2k}}_{(2k-2)}, \] with $\widetilde I_{(0)}=1$. The hat indicates that the corresponding index is omitted. The recurrence follows because the first index must pair with one of the remaining $2k-1$ indices; contraction gives $n$ for the pairing with the originally paired index and $1$ for each other pairing. This yields the factor $1/(n+2k-2)$ [1701.04549]. At low ranks, \[ \widetilde I^{ij}_{(2)}=\frac{\delta_{ij}}{n}, \] and \[ \widetilde I^{ijkl}_{(4)} = \frac{ \delta_{ij}\delta_{kl} +\delta_{ik}\delta_{jl} +\delta_{il}\delta_{jk} }{n(n+2)}. \] Thus, for example, \[ \widetilde I^{ijkl}_{(4)}a_i a_j b_k b_l = \frac{a^2b^2+2(a\cdot b)^2}{n(n+2)}. \] ## 3. Angular averages, tensor integrals, and subspaces Let $\hat r$ be a unit vector in $n$ dimensions. Its normalized angular moments are totally symmetric and isotropic, so \[ \left\langle \hat r_{i_1}\cdots\hat r_{i_{2k}} \right\rangle_{\hat r} = \widetilde I^{i_1\cdots i_{2k}}_{(2k)}, \] whereas all odd moments vanish: \[ \left\langle \hat r_{i_1}\cdots\hat r_{i_{2k+1}} \right\rangle_{\hat r}=0. \] The solid angle is \[ \Omega^{(n)}=\frac{2\pi^{n/2}}{\Gamma(n/2)}. \] For a constant vector $a$, \[ \left\langle(a\cdot\hat r)^{2k+1}\right\rangle_{\hat r}=0, \] and \[ \left\langle(a\cdot\hat r)^{2k}\right\rangle_{\hat r} = |a|^{2k} \frac{\Gamma(n/2)\Gamma(k+1/2)} {\sqrt{\pi}\,\Gamma(n/2+k)} = |a|^{2k} \frac{(2k-1)!!}{n(n+2)\cdots(n+2k-2)}. \] For a scalar function $f(q)$ depending only on $q^2$, rotational symmetry gives \[ A^{i_1\cdots i_{2k}}_{(2k)} = \widetilde I^{i_1\cdots i_{2k}}_{(2k)} \int d^nq\,(q^2)^k f(q), \] while odd-rank tensor integrals vanish. Hence all tensor components are reduced to one scalar integral multiplied by the universal isotropic tensor. The same structure applies when rotational symmetry is restricted to a subspace. If \[ V=V_{\parallel}\oplus V_{\perp}, \] with transverse projector $\delta_{\perp}^{ij}$, then \[ \widetilde I^{i_1\cdots i_k}_{\perp(k)} = \left. \widetilde I^{i_1\cdots i_k}_{(k)} \right|_{\delta^{ij}\to\delta_{\perp}^{ij},\;n\to n-m}, \] where $m$ is the number of longitudinal directions. For one distinguished direction, the transverse rank-four tensor is \[ \widetilde I^{ijkl}_{\perp(4)} = \frac{ \delta^\perp_{ij}\delta^\perp_{kl} +\delta^\perp_{ik}\delta^\perp_{jl} +\delta^\perp_{il}\delta^\perp_{jk} }{(n-1)(n+1)}. \] In dimensional regularization, the dimension is analytically continued through \[ d=4-2\epsilon,\qquad n=d-1=3-2\epsilon. \] The factors $n(n+2)\cdots(n+2k-2)$ and $\Gamma(n/2)$ are continued to noninteger dimension. In Minkowski space, the metric replaces the Euclidean delta: \[ \delta_{ij}\longrightarrow g_{\mu\nu},\qquad n\longrightarrow d. \] For a transverse subspace, \[ g_{\perp}^{\mu\nu} = g^{\mu\nu} -\frac{a^\mu a^\nu}{a\cdot a}, \] and $d$ is replaced by $d-1$ in the corresponding normalization. ## 4. Multiway covariance and the implicit delta The array-normal papers do not introduce a standalone theorem explicitly named the Kronecker Delta Rule. Their displayed formulas use Kronecker products and the reversed-order notation $\otimes^i$. The ordinary delta arises implicitly through identity covariance matrices. For an order-$i$ array $\widetilde X$ with dimensions \[ m_1\times m_2\times\cdots\times m_i, \] the vectorization operator stacks its $m_1m_2\cdots m_i$ entries: \[ rvec(\widetilde X)=x. \] If $\iota(a)$ denotes the scalar vectorization index corresponding to a multi-index $a$, then \[ \operatorname{Cov}\bigl((\widetilde X)_a,(\widetilde X)_b\bigr) = \operatorname{Cov}\bigl(x_{\iota(a)},x_{\iota(b)}\bigr). \] For a standardized array with identity covariance, \[ rvec(\widetilde Z)\sim N_m(0,I_m), \] the vectorized covariance is \[ \operatorname{Cov}(x_a,x_b)=\delta_{ab}. \] Retaining the multiway indices gives \[ \operatorname{Cov} \left( \widetilde Z_{r_1\ldots r_i}, \widetilde Z_{s_1\ldots s_i} \right) = \delta_{\iota(r_1,\ldots,r_i),\iota(s_1,\ldots,s_i)} = \prod_{k=1}^i\delta_{r_ks_k}. \] Thus the covariance is one exactly when every mode index agrees. Mode-wise matrix multiplication is defined by \[ \left((A_1)^1\cdots(A_i)^i\widetilde X\right)_{q_1\ldots q_i} = \sum_{r_1,\ldots,r_i} (A_1)_{q_1r_1}\cdots(A_i)_{q_ir_i} \widetilde X_{r_1\ldots r_i}. \] Vectorization converts this operation into \[ rvec(\widetilde L) = (A_1\otimes^i\cdots\otimes^i A_i)rvec(\widetilde X). \] When every mode matrix is an identity, repeated delta contraction yields \[ \sum_{r_1,\ldots,r_i} \delta_{q_1r_1}\cdots\delta_{q_ir_i} X_{r_1\ldots r_i} = X_{q_1\ldots q_i}. \] The general covariance construction is \[ \widetilde X = (A_1)^1\cdots(A_i)^i\widetilde Z+\widetilde M, \] with nonsingular $A_j$. Writing \[ L=A_1\otimes^i\cdots\otimes^i A_i, \] one obtains \[ \operatorname{Cov}(rvec(\widetilde X)) = LL' = (A_1A_1')\otimes^i\cdots\otimes^i(A_iA_i'). \] If $\Sigma_j=A_jA_j'$, then \[ \Lambda=\Sigma_1\otimes^i\cdots\otimes^i\Sigma_i, \] and, elementwise, \[ \operatorname{Cov} \left( X_{r_1\ldots r_i},X_{s_1\ldots s_i} \right) = \prod_{k=1}^i(\Sigma_k)_{r_ks_k}. \] The identity-delta covariance is only the special case $\Sigma_k=I_{m_k}$ [1103.3543]. This terminology must not be interpreted as an assumption of complete independence. The factors $\Sigma_k$ may contain arbitrary variances and covariances subject to the stated positive-definiteness and rank conditions. “Kronecker delta covariance structure” refers primarily to separability across array modes, not to a diagonal covariance matrix [1104.1767]. ## 5. Slicing and structured high-dimensional covariance estimation Slicing reshapes a $p$-dimensional vector into an array by choosing \[ p=m_1m_2\cdots m_i. \] The covariance is then modeled as \[ \operatorname{Cov}(x) = \Sigma_1\otimes^i\cdots\otimes^i\Sigma_i. \] The choice of factorization and variable ordering is part of the model. A 24-dimensional vector can, for example, be organized as \[ 2\times12,\qquad 3\times8,\qquad 4\times6,\qquad 2\times3\times4, \] and different organizations imply different covariance structures. For independent observations $\widetilde X_1,\ldots,\widetilde X_N$, the mean estimator is \[ \widehat{\widetilde M} = \frac1N\sum_{\ell=1}^N\widetilde X_\ell. \] The mode-specific covariance factors are estimated by an alternating procedure. At each mode, the remaining modes are whitened using current factor estimates, a covariance is estimated along the selected dimension, a square-root factor is obtained, and the factor is rescaled according to normalization constraints. The updates are repeated until convergence. The parameterization is not identifiable without normalization, because a scalar can be transferred from one factor to another without changing the total Kronecker product. The paper imposes normalization conditions requiring specified diagonal elements of $A_jA_j'$ to equal $1$. The sample-size condition is \[ N\prod_{j=1}^i m_j>m_r^2,\qquad r=1,\ldots,i, \] or equivalently \[ Np>m_r^2 \] for every mode. This is weaker than $N>p$, which is required for nonsingularity of the unrestricted sample covariance. The estimated covariance is \[ \widehat\Lambda = (\widehat A_1\otimes^i\cdots\otimes^i\widehat A_i) (\widehat A_1\otimes^i\cdots\otimes^i\widehat A_i)'. \] If the factor estimates are positive definite, then \[ \widehat\Lambda\succ0. \] An unrestricted $p\times p$ covariance requires approximately $p(p+1)/2$ parameters, whereas a separable covariance requires approximately \[ \sum_{r=1}^i\frac{m_r(m_r+1)}2 \] factor parameters, minus normalization constraints. The Kronecker representation also factorizes the eigenstructure. Eigenvalues of \[ \Lambda=\Sigma_1\otimes^i\cdots\otimes^i\Sigma_i \] are products of mode-specific eigenvalues, and corresponding eigenvectors are Kronecker products of mode-specific eigenvectors. Under misspecification, the estimator is not generally consistent for an arbitrary unrestricted covariance, although it can function as a structured regularizer with decreasing variance and residual bias. In the Alon colon gene-expression example, 2000-gene observations from 62 samples were reshaped into $40\times50$ arrays. The resulting covariance estimate was used in an $F^+$ test and Fisher’s linear discriminant analysis. The reported values were \[ F^+=3023.273,\qquad p=0.01445, \] with a reported misclassification rate of $11.3\%$ [1104.1767]. ## 6. Kronecker coefficients and the Kronecker Delta Rule in linear attention ### Kronecker coefficients The “Kronecker rule” in algebraic combinatorics is unrelated to the Kronecker delta. For partitions $\lambda,\mu,\nu\vdash n$, the Kronecker coefficient is \[ g_{\lambda\mu\nu} = \dim\operatorname{Hom}_{S_n}(V_\nu,V_\lambda\otimes V_\mu). \] For a hook \[ \mu(d)=(n-d,1^d), \] the simplified rule counts colored tableaux. Using the natural order \[ 1<\overline1<2<\overline2<\cdots