Online Variational Sequential Monte Carlo (2312.12616v3)
Abstract: Being the most classical generative model for serial data, state-space models (SSM) are fundamental in AI and statistical machine learning. In SSM, any form of parameter learning or latent state inference typically involves the computation of complex latent-state posteriors. In this work, we build upon the variational sequential Monte Carlo (VSMC) method, which provides computationally efficient and accurate model parameter estimation and Bayesian latent-state inference by combining particle methods and variational inference. While standard VSMC operates in the offline mode, by re-processing repeatedly a given batch of data, we distribute the approximation of the gradient of the VSMC surrogate ELBO in time using stochastic approximation, allowing for online learning in the presence of streams of data. This results in an algorithm, online VSMC, that is capable of performing efficiently, entirely on-the-fly, both parameter estimation and particle proposal adaptation. In addition, we provide rigorous theoretical results describing the algorithm's convergence properties as the number of data tends to infinity as well as numerical illustrations of its excellent convergence properties and usefulness also in batch-processing settings.
- Bishop, C. M. (2016). Pattern Recognition and Machine Learning. Springer.
- Importance weighted autoencoders. In International Conference on Learning Representations.
- Cappé, O. (2011). Online EM algorithm for hidden Markov models. J. Comput. Graph. Statist., 20(3):728–749.
- Inference in Hidden Markov Models. Springer, New York.
- An introduction to sequential Monte Carlo methods. Springer, New York.
- A recurrent latent variable model for sequential data. In Advances in Neural Information Processing Systems, volume 28, pages 2980 – 2988. Curran Associates, Inc.
- Adaptive methods for sequential importance sampling with application to state space models. Stat. Comput., 18(4):461–480.
- Adaptive sequential Monte Carlo by means of mixture of experts. Stat. Comput., 24(3):317–337.
- Del Moral, P. (2004). Feynman–Kac Formulae. Genealogical and Interacting Particle Systems with Applications. Springer, New York.
- Uniform stability of a particle approximation of the optimal filter derivative. SIAM Journal on Control and Optimization, 53(3):1278–1304.
- Markov Chains. Spinger.
- Sequential Monte Carlo Methods in Practice. Springer, New York.
- On sequential Monte-Carlo sampling methods for Bayesian filtering. Stat. Comput., 10:197–208.
- Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F, Radar Signal Process., 140:107–113.
- Neural adaptive sequential Monte Carlo. Advances in neural information processing systems, 28.
- The pricing of options on assets with stochastic volatilities. J. Finance, 42:281–300.
- On particle methods for parameter estimation in state-space models. Statistical science, 30(3):328–351.
- Non-asymptotic analysis of biased stochastic approximation scheme. In Conference on Learning Theory, pages 1944–1974. PMLR.
- Adam: A method for stochastic optimization. In International Conference on Learning Representations.
- Auto-encoding variational Bayes. In International Conference on Learning Representations.
- Kitagawa, G. (1996). Monte-Carlo filter and smoother for non-Gaussian nonlinear state space models. J. Comput. Graph. Statist., 1:1–25.
- Auto-encoding sequential Monte Carlo. In International Conference on Learning Representations.
- Recursive estimation in HMMs. In Proc. IEEE Conf. Decis. Control, pages 3468–3473.
- Filtering variational objectives. Advances in Neural Information Processing Systems, 30.
- Markov Chains and Stochastic Stability. Cambridge University Press, London.
- Online learning with hidden Markov models. Neural Computation, 20(7):1706–1716.
- Variational sequential Monte Carlo. In International conference on artificial intelligence and statistics, pages 968–977. PMLR.
- Particle-based online estimation of tangent filters with application to parameter estimation in nonlinear state-space models. Annals of the Institute of Statistical Mathematics, 72:545–576.
- Particle approximations of the score and observed information matrix in state space models with application to parameter estimation. Biometrika, 98(1):65–80.
- A stochastic approximation method. Ann. Math. Statist., 22:400–407.
- Asymptotic properties of recursive maximum likelihood estimation in non-linear state-space models. arXiv preprint arXiv:1806.09571.
- Streaming variational monte carlo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1):1150–1161.