Learning Optimal Filters Using Variational Inference (2406.18066v2)

Abstract: Filtering - the task of estimating the conditional distribution of states of a dynamical system given partial, noisy, observations - is important in many areas of science and engineering, including weather and climate prediction. However, the filtering distribution is generally intractable to obtain for high-dimensional, nonlinear systems. Filters used in practice, such as the ensemble Kalman filter (EnKF), are biased for nonlinear systems and have numerous tuning parameters. Here, we present a framework for learning a parameterized analysis map - the map that takes a forecast distribution and observations to the filtering distribution - using variational inference. We show that this methodology can be used to learn gain matrices for filtering linear and nonlinear dynamical systems, as well as inflation and localization parameters for an EnKF. Future work will apply this framework to learn new filtering algorithms.