Stochastic approximation in infinite dimensions
Abstract: Stochastic Approximation (SA) was introduced in the early 1950's and has been an active area of research for several decades. While the initial focus was on statistical questions, it was seen to have applications to signal processing, convex optimisation. %Over the last decade, there has been a revival of interest in SA as In later years SA has found application in Reinforced Learning (RL) and led to revival of interest. While bulk of the literature is on SA for the case when the observations are from a finite dimensional Euclidian space, there has been interest in extending the same to infinite dimension. Extension to Hilbert spaces is relatively easier to do, but this is not so when we come to a Banach space - since in the case of a Banach space, even {\em law of large numbers} is not true in general. We consider some cases where approximation works in a Banach space. Our framework includes case when the Banach space $\Bb$ is $\Cb([0,1],\Rd)$, as well as $\L1([0,1],\Rd)$, the two cases which do not even have the Radon-Nikodym property.
- Adaptive Algorithms and Stochastic Approximation. Springer-Verlag, 1990.
- Bertsekas., D. P.: Reinforcement Learning and Optimal Control. Athena Scientific, 2019.
- Blum, J. R.: Multivariable stochastic approximation methods. Annals of Mathematical Statistics, 25(4) : 737–744, 1954.
- Brown., B. M.: A General Three-Series Theorem. Proceedings of the American Mathematical Society, 28(2) : 573–577, 1971.
- Borkar.,V. S.: Asynchronous stochastic approximations. SIAM Journal on Control and Optimization, 36(3) : 840–851, 1998.
- Borkar.,V. S.: Stochastic Approximation: A Dynamical Systems Viewpoint. Hindustan Book Agency, New Delhi, India and Cambridge University Press, Cambridge, UK, 2008.
- Borkar.,V. S. and Meyn., S.P.: The O.D.E. method for convergence of stochastic approximation and reinforcement learning. SIAM Journal on Control and Optimization, 38 : 447–469, 2000.
- Dieuleveut, A.: Stochastic approximation in Hilbert spaces. Statistics [math.ST]. Universite Paris sciences et lettres, 2017.
- Stochastic approximation properties in Banach spaces. Studia Mathematica, 159(1) : 103-119, 2003.
- Convergence of stochastic iterative dynamic programming algorithms. Neural Computation, 6 : 1185–1201, 1994.
- Convergence of batch asynchronous stochastic approximation with applications to reinforcement learning. https://arxiv.org/pdf/2109.03445.pdf (2021).
- Convergence Rates for Stochastic Approximation: Biased Noise with Unbounded Variance, and Applications. https://arxiv.org/pdf/2312.02828v2.pdf (2024)
- Stochastic Approximation in Hilbert Space: Identification and Optimization of Linear Continuous Parameter Systems. SIAM Journal on Control and Optimization, 23(5) : 774–793, 1985.
- Ledoux, M. and Talagrand, M.: Probability in Banach Spaces: Isoperimetry and Processes. Springer-Verlag, 1991.
- Lai, T. L. : Stochastic approximation (invited paper). The Annals of Statistics, 31(2) : 391– 406, 2003.
- Milz, J.: Sample average approximations of strongly convex stochastic programs in Hilbert spaces. Optim Lett 17 : 471–492, 2023. https://doi.org/10.1007/s11590-022-01888-4
- A stochastic approximation method. Annals of Mathematical Statistics, 22(3) : 400–407, 1951.
- Woyczynski, W. A.: Geometry and martingales in Banach spaces. CRC Press, 2019.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.