Constrained and Preconditioned Stochastic Gradient Method

Abstract: We consider stochastic approximations which arise from such applications as data communications and image processing. We demonstrate why constraints are needed in a stochastic approximation and how a constrained approximation can be incorporated into a preconditioning technique to derive the pre-conditioned stochastic gradient method (PSGM). We perform convergence analysis to show that the PSGM converges to the theoretical best approximation under some simple assumptions on the preconditioner and on the independence of samples drawn from a stochastic process. Simulation results are presented to demonstrate the effectiveness of the constrained and precondi-tioned stochastic gradient method.