Neural Bayesian Filtering (NBF)

• Neural Bayesian Filtering is a family of methods that combines learned neural representations with Bayesian filtering to update latent state estimates in dynamic, partially observable systems.
• The 2025 formulation introduces particle-style updates in an embedding space and proves convergence at the Monte Carlo rate, O_p(n⁻¹/²), under ideal conditions.
• NBF encompasses diverse architectures—discriminative, recurrent, and flow-based filters—that enable applications in neural decoding, forecasting, and anomaly detection.

Neural Bayesian Filtering (NBF) denotes a family of methods that maintain a posterior distribution, or belief state, in a partially observable system while replacing some part of the classical filtering pipeline with learned neural or latent-variable machinery. In the specific usage formalized by "Neural Bayesian Filtering" (Solinas et al., 4 Oct 2025), beliefs are mapped to fixed-length embedding vectors that condition a generative model and are updated by particle-style Bayesian steps in embedding space. In the broader literature, closely related methods include discriminative filters that model $p(z_t\mid x_t)$ rather than $p(x_t\mid z_t)$, recurrent architectures that align separate neural modules with prediction and correction, flow-based filters that make latent filtering Gaussian through invertible transformations, BSDE-based filters that learn the filtering density itself, and parameter-space filters that recursively infer neural-network weights rather than external latent states (Burkhart, 2018, Lim et al., 2019, Wang et al., 22 Feb 2025, Bågmark et al., 14 Aug 2025, Duran-Martin et al., 2021).

1. Filtering problem and conceptual scope

At its core, NBF inherits the standard Bayesian filtering objective: recursively estimate a latent quantity from noisy observations. In state-space notation this is the recursion

$$p(x_t,\theta\mid y_{1:t-1}) = \int p(x_t\mid x_{t-1},\theta)\, p(x_{t-1},\theta\mid y_{1:t-1})\, dx_{t-1},$$

followed by

$$p(x_t,\theta\mid y_{1:t}) \propto p(y_t\mid x_t,\theta)\, p(x_t,\theta\mid y_{1:t-1}).$$

In the classical setting, the main design choices are the transition model, the observation model, and the posterior family. NBF keeps this recursive structure but changes how one or more of these objects are represented (Lin et al., 2017).

The literature uses the label in more than one way. In one line of work, the latent variable is the hidden state of an external dynamical system, and neural components are used to represent beliefs, transition laws, observation conditionals, or recursive filtering operators. In another line, the latent quantity is not an external state but a neural-network parameter, a low-dimensional weight subspace coordinate, or even the moving optimum of a parameter during training; the observations are then rewards or gradients, and the resulting recursion is still a Bayesian filter in parameter space (Aitchison, 2018, Bencomo et al., 2023).

A common misconception is that NBF must mean end-to-end deep sequence models trained to output posterior states directly. The literature is broader. Some methods retain explicit Bayesian updates and only learn an observation-side inverse conditional; some learn a latent representation in which filtering becomes Gaussian; some are only “neural” in application domain or in one component of the filter; and some relevant antecedents are not neural in the modern deep-learning sense at all (Burkhart, 2018, Wang et al., 22 Feb 2025, Lin et al., 2017).

2. The explicit 2025 formulation of Neural Bayesian Filtering

The paper "Neural Bayesian Filtering" defines NBF for partially observable systems by learning a latent representation of the family of beliefs induced by a task (Solinas et al., 4 Oct 2025). The hidden state is $x_t$, the observation is $y_t$, and the filtering target is

$p(x_t \mid y_{1:t}, \pi, G),$

given environment dynamics $G$ and control or policy information $\pi$. The induced belief family is

$$\mathcal{P} = \{p(x_t \mid y_{1:t}, \pi, G) : \pi \in \Pi,\; G \in \mathcal{G},\; y^{(i)} \in Y,\; t \in \mathbb{N}\}.$$

The central representation is a fixed-length belief embedding

$p(x_t\mid z_t)$0

computed from a set of weighted samples from a belief. Conditioned on $p(x_t\mid z_t)$1, a generative model $p(x_t\mid z_t)$2 is used for both sampling and density evaluation. In the implementation reported in the paper, the belief model is a conditional normalizing flow, so sampling proceeds by drawing $p(x_t\mid z_t)$3 and mapping $p(x_t\mid z_t)$4.

Online filtering is performed by a particle-style recursion in embedding space. From the current embedding $p(x_t\mid z_t)$5, one samples particles $p(x_t\mid z_t)$6, propagates them with the known dynamics $p(x_t\mid z_t)$7, computes weights

$p(x_t\mid z_t)$8

normalizes the weights, and re-embeds the weighted propagated particles:

$p(x_t\mid z_t)$9

The maintained state of the filter is therefore not a particle cloud and not a Gaussian moment pair, but a learned belief code from which arbitrarily many fresh particles can be generated. This is the mechanism used to mitigate, though not eliminate, particle impoverishment.

The paper also gives an idealized consistency result. Under a perfect embedding model and an $$p(x_t,\theta\mid y_{1:t-1}) = \int p(x_t\mid x_{t-1},\theta)\, p(x_{t-1},\theta\mid y_{1:t-1})\, dx_{t-1},$$0-global observation positivity condition, for any bounded function $$p(x_t,\theta\mid y_{1:t-1}) = \int p(x_t\mid x_{t-1},\theta)\, p(x_{t-1},\theta\mid y_{1:t-1})\, dx_{t-1},$$1, the NBF estimate $$p(x_t,\theta\mid y_{1:t-1}) = \int p(x_t\mid x_{t-1},\theta)\, p(x_{t-1},\theta\mid y_{1:t-1})\, dx_{t-1},$$2 converges almost surely to the true posterior expectation $$p(x_t,\theta\mid y_{1:t-1}) = \int p(x_t\mid x_{t-1},\theta)\, p(x_{t-1},\theta\mid y_{1:t-1})\, dx_{t-1},$$3 as the particle count $$p(x_t,\theta\mid y_{1:t-1}) = \int p(x_t\mid x_{t-1},\theta)\, p(x_{t-1},\theta\mid y_{1:t-1})\, dx_{t-1},$$4, with Monte Carlo rate

$$p(x_t,\theta\mid y_{1:t-1}) = \int p(x_t\mid x_{t-1},\theta)\, p(x_{t-1},\theta\mid y_{1:t-1})\, dx_{t-1},$$5

This places the method between classical parametric filters and sequential Monte Carlo: the update is particle-like, but the maintained belief is a learned latent representation rather than a weighted empirical measure.

3. Discriminative, recurrent, and hybrid neural filters

A major branch of NBF replaces the generative observation model by a discriminative conditional. The Discriminative Kalman Filter (DKF) keeps linear-Gaussian latent dynamics,

$$p(x_t,\theta\mid y_{1:t-1}) = \int p(x_t\mid x_{t-1},\theta)\, p(x_{t-1},\theta\mid y_{1:t-1})\, dx_{t-1},$$6

but models

$$p(x_t,\theta\mid y_{1:t-1}) = \int p(x_t\mid x_{t-1},\theta)\, p(x_{t-1},\theta\mid y_{1:t-1})\, dx_{t-1},$$7

Using

$$p(x_t,\theta\mid y_{1:t-1}) = \int p(x_t\mid x_{t-1},\theta)\, p(x_{t-1},\theta\mid y_{1:t-1})\, dx_{t-1},$$8

the update remains Gaussian:

$$p(x_t,\theta\mid y_{1:t-1}) = \int p(x_t\mid x_{t-1},\theta)\, p(x_{t-1},\theta\mid y_{1:t-1})\, dx_{t-1},$$9

This formulation is particularly suited to neural decoding, where observations are high-dimensional neural features and the latent state is low-dimensional kinematics; in BrainGate2, the DKF enabled three volunteers with quadriplegia to control an on-screen cursor in real time, and participant “T9” used the DKF to type out messages on a tablet PC (Burkhart, 2018). A later reproduction study confirmed the method’s practical relevance for offline neural decoding and found that DKF-NW was best on average in mean absolute angle error, while also emphasizing sensitivity to preprocessing and regression backends (Casco-Rodriguez et al., 2024).

A second branch learns the recursive filter operators themselves. Recurrent Neural Filters (RNF) decompose filtering into propagation, input dynamics, and error correction:

$$p(x_t,\theta\mid y_{1:t}) \propto p(y_t\mid x_t,\theta)\, p(x_t,\theta\mid y_{1:t-1}).$$0

The core claim is that many neural latent sequence models entangle prediction and update in a single recurrent mechanism, whereas RNF learns distinct representations for each Bayesian filtering step. Empirically, RNF improved one-step forecasting accuracy by $$p(x_t,\theta\mid y_{1:t}) \propto p(y_t\mid x_t,\theta)\, p(x_t,\theta\mid y_{1:t-1}).$$1 on average across datasets and benchmarks, and its decoupling was especially useful for multistep prediction and missing-observation settings (Lim et al., 2019).

Model-based hybrid architectures push this decomposition further. The "Explainable Gated Bayesian Recurrent Neural Network" derives a filter for non-Markov state estimation under model mismatch by introducing a memory variable and mismatch terms $$p(x_t,\theta\mid y_{1:t}) \propto p(y_t\mid x_t,\theta)\, p(x_t,\theta\mid y_{1:t-1}).$$2 and $$p(x_t,\theta\mid y_{1:t}) \propto p(y_t\mid x_t,\theta)\, p(x_t,\theta\mid y_{1:t-1}).$$3. The resulting architecture has a memory update gate, a state prediction gate with evolution mismatch compensation, and a state update gate with observation mismatch compensation. Under a Gaussian approximation, the learned mismatch means and covariances enter Kalman-like recursions directly, and ablations show that removing any of the three gates degrades performance materially (Yan et al., 2023).

A closely related applied instance is NSIBF, which first learns a nonlinear state-space model from normal data using an encoder $$p(x_t,\theta\mid y_{1:t}) \propto p(y_t\mid x_t,\theta)\, p(x_t,\theta\mid y_{1:t-1}).$$4, transition model $$p(x_t,\theta\mid y_{1:t}) \propto p(y_t\mid x_t,\theta)\, p(x_t,\theta\mid y_{1:t-1}).$$5, and decoder $$p(x_t,\theta\mid y_{1:t}) \propto p(y_t\mid x_t,\theta)\, p(x_t,\theta\mid y_{1:t-1}).$$6,

$$p(x_t,\theta\mid y_{1:t}) \propto p(y_t\mid x_t,\theta)\, p(x_t,\theta\mid y_{1:t-1}).$$7

and then applies a UKF-style Gaussian filter on top of the learned model. The anomaly score is a Mahalanobis distance in predicted observation space, and the paper reports improvements over the second-best methods of $$p(x_t,\theta\mid y_{1:t}) \propto p(y_t\mid x_t,\theta)\, p(x_t,\theta\mid y_{1:t-1}).$$8 on PUMP, $$p(x_t,\theta\mid y_{1:t}) \propto p(y_t\mid x_t,\theta)\, p(x_t,\theta\mid y_{1:t-1}).$$9 on WADI, and $x_t$0 on SWAT (Feng et al., 2021).

4. Latent-variable, flow-based, and density-learning approaches

One important antecedent to modern NBF is EnLLVM, an ensemble-based nonlinear Bayesian filter built around linear latent variable models rather than deep networks. It approximates the predictive joint density $x_t$1 by a Gaussian mixture obtained from ensemble-member-wise latent reconstructions,

$x_t$2

and then conditions each component on the new observation. The method is not neural in the modern sense, but it is conceptually close to later NBF work because it performs filtering via a learned lower-dimensional latent representation, avoids explicit likelihood evaluation in the original coordinates, and can handle intractable likelihoods through simulated observation samples and a surrogate joint model (Lin et al., 2017).

Flow-based Bayesian filtering moves this idea into a deep invertible setting. The Flow-based Bayesian Filter (FBF) learns invertible maps

$x_t$3

and constructs a latent state-space model in which the filtering law remains Gaussian:

$x_t$4

If

$x_t$5

then the updated latent posterior remains Gaussian and the original-space filtering density is recovered exactly by change of variables:

$x_t$6

This lets the method represent highly non-Gaussian posteriors in the original state space while retaining analytic recursive inference in latent space (Wang et al., 22 Feb 2025).

A more direct density-based route is taken by "Nonlinear filtering based on density approximation and deep BSDE prediction." That paper considers continuous-time latent dynamics with discrete-time observations,

$x_t$7

and primarily targets an unnormalized filtering density. Between observations, the density evolves by the Fokker–Planck equation, and the prediction step is rewritten through a nonlinear Feynman–Kac representation as an uncoupled FBSDE. The learned object is a sequence of neural functions $x_t$8 approximating the predicted unnormalized density, with recursive update

$x_t$9

This is a strict density-learning version of NBF: it does not merely propagate moments or samples, but learns a recursive approximation of the filtering density itself (Bågmark et al., 14 Aug 2025).

5. Filtering in neural-network parameter space

Several papers instantiate Bayesian filtering not over an external hidden state but over neural-network parameters or parameter-dependent latent quantities. In "Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods," the latent state is the coordinate-wise moving optimum

$y_t$0

the observation is the backpropagated gradient $y_t$1, and the resulting scalar linear-Gaussian filter yields AdaBayes. Under the derived asymptotics, the posterior variance interpolates between SGD-like and Adam/RMSprop-like behavior, and the high-information limit produces an AdamW-like update with decoupled weight decay (Aitchison, 2018).

Kalman Bayesian Neural Networks likewise cast neural learning as filtering and smoothing. The primary filtered state is the collection of layer weights $y_t$2, each neuron’s weight vector is Gaussian, and the backward pass uses Kalman or Rauch–Tung–Striebel-style equations to update both hidden quantities and weights in closed form. This is not a latent-state filter in the usual dynamical-systems sense, but it is a sample-by-sample Bayesian filter over neural parameters without gradient descent (Wagner et al., 2021).

The neural bandits literature pushes this idea into high dimensions by combining an EKF with a low-dimensional affine parameter subspace

$y_t$3

so that recursive Bayesian inference is performed over $y_t$4 rather than the full weight vector. The latent transition is a random walk,

$y_t$5

the observation model is a neural reward predictor, and the posterior is updated by EKF in the learned or random subspace. The method uses constant memory and represents uncertainty over all model parameters, not just the final linear layer (Duran-Martin et al., 2021).

A related optimization-based formulation is "Implicit Maximum a Posteriori Filtering via Adaptive Optimization." There the filter is specified implicitly by an optimizer run for $y_t$6 steps on the time-varying MAP objective

$y_t$7

rather than by explicit covariance recursions. In the linear-Gaussian setting, every Kalman filter has an equivalent formulation using $y_t$8 steps of gradient descent, and in a high-dimensional Yearbook experiment the latent state dimension was $y_t$9, illustrating the intended regime of scalable parameter-space filtering (Bencomo et al., 2023).

6. Approximation theory, convergence, and stability

The theoretical foundations of NBF are heterogeneous, but several papers make explicit approximation claims. "Recurrent Neural Networks and Universal Approximation of Bayesian Filters" proves that, on a fixed finite horizon, a generic deep RNN can approximate the Bayes-optimal estimator arbitrarily well in average $p(x_t \mid y_{1:t}, \pi, G),$0 error:

$p(x_t \mid y_{1:t}, \pi, G),$1

Under additional assumptions—stationarity of observations and exponential stability of the true filter—it further gives a time-uniform result: there exists an RNN whose error remains uniformly bounded for all $p(x_t \mid y_{1:t}, \pi, G),$2. This result is theoretical support for learning the recursive filtering map directly from signal-observation pairs without explicit knowledge of the transition kernel $p(x_t \mid y_{1:t}, \pi, G),$3 or observation density $p(x_t \mid y_{1:t}, \pi, G),$4 (Bishop et al., 2022).

The explicit 2025 NBF paper provides a different kind of guarantee. Its convergence statement is not about universal approximation of arbitrary filters, but about the particle-style embedding-space recursion under a perfect belief model and $p(x_t \mid y_{1:t}, \pi, G),$5-global observation positivity. In that idealized regime, NBF has standard Monte Carlo consistency and $p(x_t \mid y_{1:t}, \pi, G),$6 convergence of posterior expectations as the number of particles used in the update grows (Solinas et al., 4 Oct 2025).

The BSDE density filter provides the most explicit error decomposition. Its main theorem gives a mixed a priori–a posteriori bound of the form

$p(x_t \mid y_{1:t}, \pi, G),$7

where $p(x_t \mid y_{1:t}, \pi, G),$8 is the discretization error from the Euler–Maruyama and BSDE approximation, and the residual terms are the empirical deep-BSDE training errors. This is notable because it separates numerical time-discretization error from neural approximation and optimization error within a recursive filter (Bågmark et al., 14 Aug 2025).

These results point to three distinct theoretical perspectives on NBF. One perspective treats recurrent networks as universal approximators of Bayesian filter operators. A second treats belief embeddings plus particle-style Bayes updates as a Monte Carlo method over a learned belief manifold. A third treats density propagation as a PDE or BSDE problem approximated by neural networks. The common thread is that the learned component is inserted into a recursive Bayesian estimator rather than replacing recursion altogether.

7. Applications, empirical patterns, and limitations

The application range of NBF is broad. Neural decoding and brain-computer interfaces are a recurrent theme: the DKF was designed for high-dimensional neural recordings with low-dimensional latent motor variables, and the original DKF study reported real-time BrainGate2 control, while the reproduction study found that DKF-NW was best on average in mean absolute angle error and that DKF filtering can improve directional accuracy even when Euclidean error is not uniformly best (Burkhart, 2018, Casco-Rodriguez et al., 2024). Time-series forecasting is another major setting: RNF was evaluated on Electricity, Volatility, and Quote, and its decoupled filtering stages improved both one-step and multistep prediction (Lim et al., 2019). CPS anomaly detection provides a model-based industrial application, where NSIBF uses a learned nonlinear state-space model plus UKF-style filtering and reports large gains over residual-only ablations and strong baselines on PUMP, WADI, and SWAT (Feng et al., 2021).

High-dimensional nonlinear dynamics are a second major empirical regime. EnLLVM was compared to EnKF on Lorenz systems and was most notable for robustness under model mismatch rather than for dominance in the perfectly specified case (Lin et al., 2017). FBF was tested on a synthetic nonlinear system, high-dimensional stochastic Lorenz-96, and a stochastic advection-diffusion system, with the strongest reported gains appearing in higher dimensions where particle methods degenerate and simpler Gaussian neural filters degrade (Wang et al., 22 Feb 2025). The explicit NBF paper evaluated belief tracking in Gridworld, Goofspiel, and Triangulation, emphasizing multimodal beliefs and changing known environments or policies (Solinas et al., 4 Oct 2025).

The limitations are equally varied. Some methods are only neural in a restricted sense: EnLLVM is a classical linear latent-variable model rather than a deep neural filter (Lin et al., 2017). Several architectures rely on Gaussian approximations to latent posteriors or mismatch terms, including DKF, EGBRNN, and NSIBF, which can be restrictive when conditionals are strongly multimodal or skewed (Burkhart, 2018, Yan et al., 2023, Feng et al., 2021). FBF requires invertible transformations and supervised state-observation trajectories for training, and its latent Gaussianity remains a modeling assumption even though the original-space posterior is non-Gaussian (Wang et al., 22 Feb 2025). The BSDE density filter is mathematically structured but currently demonstrated only in one-dimensional experiments and under ellipticity and smoothness assumptions (Bågmark et al., 14 Aug 2025). The 2025 NBF framework depends on the quality of the learned belief model and on representative training beliefs, and its online update remains particle-based even though the maintained object is an embedding (Solinas et al., 4 Oct 2025).

Taken together, these works show that NBF is not a single architecture but a design space. Some methods learn $p(x_t \mid y_{1:t}, \pi, G),$9; some learn the recursive operators $G$0; some learn latent memory and mismatch corrections on top of nominal Bayesian recursions; some learn invertible representations that make latent filtering Gaussian; some learn the filtering density itself; and some apply the filtering viewpoint to neural-network parameters rather than external dynamical states. The unifying idea is that Bayesian recursion remains central, but the objects being propagated, approximated, or parameterized are learned.