Closed-form Filtering for Non-linear Systems (2402.09796v1)

Abstract: Sequential Bayesian Filtering aims to estimate the current state distribution of a Hidden Markov Model, given the past observations. The problem is well-known to be intractable for most application domains, except in notable cases such as the tabular setting or for linear dynamical systems with gaussian noise. In this work, we propose a new class of filters based on Gaussian PSD Models, which offer several advantages in terms of density approximation and computational efficiency. We show that filtering can be efficiently performed in closed form when transitions and observations are Gaussian PSD Models. When the transition and observations are approximated by Gaussian PSD Models, we show that our proposed estimator enjoys strong theoretical guarantees, with estimation error that depends on the quality of the approximation and is adaptive to the regularity of the transition probabilities. In particular, we identify regimes in which our proposed filter attains a TV $\epsilon$-error with memory and computational complexity of $O(\epsilon^{-1})$ and $O(\epsilon^{-3/2})$ respectively, including the offline learning step, in contrast to the $O(\epsilon^{-2})$ complexity of sampling methods such as particle filtering.

• The paper introduces a novel closed-form filtering algorithm based on Gaussian PSD Models, enabling efficient sequential Bayesian inference with strong theoretical guarantees.
• It demonstrates that the proposed PSDFilter and its generalizations reduce computational complexity while robustly approximating smooth transition densities compared to traditional methods.
• The methodology extends the scope of Kalman filters by approximating non-linear dynamics and learning model parameters via non-convex optimization techniques.

Expanding the Utility of Gaussian PSD Models for Non-linear Bayesian Filtering

Introduction to Gaussian PSD Models

Gaussian PSD Models, proposed by Rudi and Ciliberto (2021), provide a flexible framework for modeling probabilistic densities, significantly broadening the scope of Gaussian Mixture Models by allowing for negative coefficients within the mixture. This capability renders them particularly useful in applications requiring Bayesian inference, notably in the domain of sequential Bayesian Filtering. These models can optimally approximate a broad range of probability densities with theoretical guarantees, and uniquely, operations like product, marginalization, and integration can be performed in a closed-form manner.

Analytical Advancements in Non-linear Filtration

The paper presents an innovative class of filters based on Gaussian PSD Models that enhance potential applications in density approximation and computational efficiency. Through a novel algorithm, it tackles the Sequential Bayesian Filtering problem by approximating the unknown transition and observation probabilities using Gaussian PSD Models. This methodology not only extends the purview of estimators such as the Kalman filter but also establishes strong theoretical guarantees in diverse settings.

Theoretical Framework and Algorithm Proposition

A primary contribution of this work is the formulation of a novel algorithm for Sequential Bayesian Filtering, which is notably applicable to any filtering problem where transition kernels possess a smooth density. The paper delineates the Gaussian PSD Models and their properties, particularly emphasizing their stability concerning probabilistic operations. It further outlines an algorithm for learning these models from function evaluations, achieving optimal estimation rates for smooth targets.

Prospects of Approximation and Filtering Using Gaussian PSD Models

In deriving a robust approximate filtering algorithm, referred to as PSDFilter, the paper introduces Gaussian PSD Models as approximators for transition and observation probabilities. It is demonstrated that PSDFilter maintains robustness against initial distribution choices and approximation errors in Q and G. The paper estimates that the algorithm exhibits significantly lower computational and space complexity compared to traditional methods such as particle filtering, especially when handling very regular kernels.

Beyond Gaussian PSD Models: Towards Generalization

Extending the scope of Gaussian PSD Models, the paper ventures into the field of Generalized Gaussian PSD Models. This extension facilitates handling non-diagonal precision matrices, enabling a more nuanced approximation of transition kernels. Through theoretical analysis, it is shown that Generalized Gaussian PSD Models not only inherit the desirable properties of Gaussian PSD Models but also encapsulate Kalman filters as a special case. The methodology for learning and applying these generalized models is articulated, emphasizing non-convex optimization techniques for practical implementation.

Conclusion and Implications for Future Research

The examination of Gaussian and Generalized Gaussian PSD Models in this paper marks a significant advance in the quest for efficient and effective non-linear filtering solutions. The established theoretical framework, combined with the proposed algorithmic strategies, showcases a notable enhancement in the approximation quality, computational efficiency, and applicability to a wider range of systems. These advancements promise impactful implications for future developments in AI, particularly in improving the accuracy and feasibility of sequential Bayesian filtering in complex systems. As this research front progresses, further exploration into optimizing model learning processes and expanding the utility of these models in varied applications is anticipated.