2000 character limit reached
Stochastic Recursive Gradient Algorithm for Nonconvex Optimization (1705.07261v1)
Published 20 May 2017 in stat.ML, cs.LG, and math.OC
Abstract: In this paper, we study and analyze the mini-batch version of StochAstic Recursive grAdient algoritHm (SARAH), a method employing the stochastic recursive gradient, for solving empirical loss minimization for the case of nonconvex losses. We provide a sublinear convergence rate (to stationary points) for general nonconvex functions and a linear convergence rate for gradient dominated functions, both of which have some advantages compared to other modern stochastic gradient algorithms for nonconvex losses.