Convolutional Bayesian Filtering (2404.00481v1)
Abstract: Bayesian filtering serves as the mainstream framework of state estimation in dynamic systems. Its standard version utilizes total probability rule and Bayes' law alternatively, where how to define and compute conditional probability is critical to state distribution inference. Previously, the conditional probability is assumed to be exactly known, which represents a measure of the occurrence probability of one event, given the second event. In this paper, we find that by adding an additional event that stipulates an inequality condition, we can transform the conditional probability into a special integration that is analogous to convolution. Based on this transformation, we show that both transition probability and output probability can be generalized to convolutional forms, resulting in a more general filtering framework that we call convolutional Bayesian filtering. This new framework encompasses standard Bayesian filtering as a special case when the distance metric of the inequality condition is selected as Dirac delta function. It also allows for a more nuanced consideration of model mismatch by choosing different types of inequality conditions. For instance, when the distance metric is defined in a distributional sense, the transition probability and output probability can be approximated by simply rescaling them into fractional powers. Under this framework, a robust version of Kalman filter can be constructed by only altering the noise covariance matrix, while maintaining the conjugate nature of Gaussian distributions. Finally, we exemplify the effectiveness of our approach by reshaping classic filtering algorithms into convolutional versions, including Kalman filter, extended Kalman filter, unscented Kalman filter and particle filter.
- University of California, Berkeley, 1966.
- Approximate inference in state-space models with heavy-tailed noise. IEEE Transactions on Signal Processing, 60(10):5024–5037, 2012.
- Alexander A Alemi. Variational predictive information bottleneck. In Symposium on Advances in Approximate Bayesian Inference, pages 1–6. PMLR, 2020.
- Optimal filtering. Courier Corporation, 2012.
- The iterated Kalman filter update as a Gauss-Newton method. IEEE Transactions on Automatic Control, 38(2):294–297, 1993.
- Predictability, complexity, and learning. Neural computation, 13(11):2409–2463, 2001.
- Huber-Based Adaptive Unscented Kalman Filter with Non-Gaussian Measurement Noise [J]. Circuits Systems and Signal Processing, 37(12):1–21, 2018.
- Generalized moving horizon estimation for nonlinear systems with robustness to measurement outliers. In 2023 American Control Conference (ACC), pages 1614–1621. IEEE, 2023.
- Maximum correntropy Kalman filter. Automatica, 76:70–77, 2017.
- Real-time solution of time-varying yau filtering problems via direct method and gaussian approximation. IEEE Transactions on Automatic Control, 64(4):1648–1654, 2018.
- Zhe Chen et al. Bayesian filtering: From Kalman filters to particle filters, and beyond. Statistics, 182(1):1–69, 2003.
- Thomas M Cover. Elements of information theory. John Wiley & Sons, 1999.
- Classical and Bayesian inference in neuroimaging: theory. NeuroImage, 16(2):465–483, 2002.
- Peter J Huber. Robust statistics, volume 523. John Wiley & Sons, 2004.
- Andrew H Jazwinski. Stochastic processes and filtering theory. Courier Corporation, 2007.
- New extension of the Kalman filter to nonlinear systems. In Signal processing, sensor fusion, and target recognition VI, volume 3068, pages 182–193. Spie, 1997.
- Unscented filtering and nonlinear estimation. Proceedings of the IEEE, 92(3):401–422, 2004.
- R.E. Kalman. A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 82(1):35–45, 1960.
- New results in linear filtering and prediction theory. 1961.
- An optimization-centric view on Bayes’ rule: Reviewing and generalizing variational inference. Journal of Machine Learning Research, 23(132):1–109, 2022.
- A Kolmogorov. Interpolation and extrapolation of stationary random sequences. Izvestiya Rossiiskoi Akademii Nauk. Seriya Matematicheskaya, 5:3, 1941.
- Structured inference networks for nonlinear state space models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
- Central limit theorem: the cornerstone of modern statistics. Korean journal of anesthesiology, 70(2):144–156, 2017.
- Shengbo Eben Li. Reinforcement learning for sequential decision and optimal control. Springer, 2023.
- Distributed Bayesian Filter Using Measurement Dissemination for Multiple Unmanned Ground Vehicles With Dynamically Changing Interaction Topologies. Journal of Dynamic Systems, Measurement, and Control, 140(3):030903, 11 2017.
- Sequential Monte Carlo methods for dynamic systems. Journal of the American statistical association, pages 1032–1044, 1998.
- Xue Luo and Stephen S-T Yau. Complete real time solution of the general nonlinear filtering problem without memory. IEEE Transactions on Automatic Control, 58(10):2563–2578, 2013.
- Robust Bayesian inference via coarsening. Journal of the American Statistical Association, 2018.
- Filtering via simulation: Auxiliary particle filters. Journal of the American statistical association, 94(446):590–599, 1999.
- A Student’s t filter for heavy tailed process and measurement noise. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 5770–5774. IEEE, 2013.
- Recursive noise adaptive Kalman filtering by variational Bayesian approximations. IEEE Transactions on Automatic control, 54(3):596–600, 2009.
- A Lyapunov function for robust stability of moving horizon estimation. IEEE Transactions on Automatic Control, 2023.
- Application of statistical filter theory to the optimal estimation of position and velocity on board a circumlunar vehicle, volume 135. National Aeronautics and Space Administration, 1962.
- Maximum Correntropy Ensemble Kalman Filter. In 2023 62nd IEEE Conference on Decision and Control (CDC), pages 8659–8664. IEEE, 2023.
- Yangtianze Tao and Stephen Shing-Toung Yau. Outlier-Robust Iterative Extended Kalman Filtering. IEEE Signal Processing Letters, 2023.
- The information bottleneck method. arXiv preprint physics/0004057, 2000.
- Norbert Wiener. Extrapolation, interpolation, and smoothing of stationary time series: with engineering applications. The MIT press, 1949.
- A robust particle filtering algorithm with non-Gaussian measurement noise using student-t distribution. IEEE Signal Processing Letters, 21(1):30–34, 2013.
- Shing-Tung Yau and Stephen S-T Yau. Real time solution of nonlinear filtering problem without memory I. Mathematical Research Letters, 7(6):671–693, 2000.
- Shing-Tung Yau and Stephen S-T Yau. Real time solution of the nonlinear filtering problem without memory II. SIAM Journal on Control and Optimization, 47(1):163–195, 2008.
- Moshe Zakai. On the optimal filtering of diffusion processes. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 11(3):230–243, 1969.
- Degradation data analysis and remaining useful life estimation: A review on Wiener-process-based methods. European Journal of Operational Research, 271(3):775–796, 2018.