A Block-Randomized Stochastic Method with Importance Sampling for CP Tensor Decomposition

Abstract: One popular way to compute the CANDECOMP/PARAFAC (CP) decomposition of a tensor is to transform the problem into a sequence of overdetermined least squares subproblems with Khatri-Rao product (KRP) structure involving factor matrices. In this work, based on choosing the factor matrix randomly, we propose a mini-batch stochastic gradient descent method with importance sampling for those special least squares subproblems. Two different sampling strategies are provided. They can avoid forming the full KRP explicitly and computing the corresponding probabilities directly. The adaptive step size version of the method is also given. For the proposed method, we present its detailed theoretical properties and comprehensive numerical performance. The results on synthetic and real data show that our method performs better than the corresponding one in the literature.