The Hanson-Wright Inequality for Random Tensors

Abstract: We provide moment bounds for expressions of the type $(X^{(1)} \otimes \dots \otimes X^{(d)})^T A (X^{(1)} \otimes \dots \otimes X^{(d)})$ where $\otimes$ denotes the Kronecker product and $X^{(1)}, \dots, X^{(d)}$ are random vectors with independent, mean 0, variance 1, subgaussian entries. The bounds are tight up to constants depending on $d$ for the case of Gaussian random vectors. Our proof also provides a decoupling inequality for expressions of this type. Using these bounds, we obtain new, improved concentration inequalities for expressions of the form $|B (X^{(1)} \otimes \dots \otimes X^{(d)})|_2$.