Recursive KalmanNet

• Recursive KalmanNet is a deep learning-augmented state estimation framework that integrates the classic predictor-corrector Kalman filter with RNNs.
• It employs dedicated RNNs for gain computation and covariance correction using a recursive generalization of Joseph’s formula to ensure mathematically consistent uncertainty propagation.
• Empirical evaluations show near-oracle performance in MSE and Mahalanobis distance, outperforming traditional Kalman filters and prior neural-aided approaches.

Recursive KalmanNet (RKN) is a deep learning-augmented state estimation framework that combines the classical predictor-corrector structure of the Kalman filter (KF) with recurrent neural networks (RNNs) to deliver accurate state estimates and provably consistent error covariances in linear dynamical systems with unknown and potentially non-Gaussian noise statistics. By propagating uncertainty using a recursive generalization of Joseph’s formula and optimizing the Gaussian negative log-likelihood, RKN addresses fundamental deficiencies of both classical KFs (which require precise knowledge of noise covariances and Gaussianity) and fully data-driven approaches that lack principled uncertainty quantification (Mortada et al., 13 Jun 2025).

1. State-Space Problem Formulation

RKN operates within the standard linear state-space model:

• State transition: $x_k = F_k x_{k-1} + v_k$
• Observation: $z_k = H_k x_k + w_k$

where $x_k \in \mathbb{R}^m$ is the hidden state, $z_k \in \mathbb{R}^n$ is the measurement, and $F_k \in \mathbb{R}^{m\times m}$ and $H_k \in \mathbb{R}^{n\times m}$ are known system matrices. The process noise $v_k$ and measurement noise $w_k$ are zero-mean, mutually independent (but otherwise arbitrary) white processes, typically with unknown, possibly heavy-tailed or multi-modal covariances $Q_k$ and $R_k$. This departs from the canonical Kalman filter’s assumption of known, independent Gaussian noise and places RKN in the broader category of data-driven filtering techniques designed for realistic, adverse scenarios (Mortada et al., 13 Jun 2025).

2. Recursive KalmanNet Architecture

The RKN framework retains the Kalman filter’s "predict–correct" recursion but replaces analytic gain computation and covariance update with two dedicated recurrent networks:

• Gain Network (RNN #1, parameters $z_k = H_k x_k + w_k$0):
  • Input features $z_k = H_k x_k + w_k$1 at each time $z_k = H_k x_k + w_k$2 include innovation $z_k = H_k x_k + w_k$3, the last gain update $z_k = H_k x_k + w_k$4, the system Jacobian $z_k = H_k x_k + w_k$5, and the measurement increment $z_k = H_k x_k + w_k$6.
  • The hidden state $z_k = H_k x_k + w_k$7 is evolved by a GRU and fully connected layers and outputs the estimated Kalman gain $z_k = H_k x_k + w_k$8.
• Covariance Network (RNN #2, parameters $z_k = H_k x_k + w_k$9):
  • Receives the same feature vector $x_k \in \mathbb{R}^m$0.
  • Outputs the Cholesky factor $x_k \in \mathbb{R}^m$1 of a learned noise-dependent covariance term $x_k \in \mathbb{R}^m$2 for correction.

The forward pass at each step $x_k \in \mathbb{R}^m$3 comprises prediction, feature formation, gain computation, partial covariance update (using Joseph’s formula), learned covariance correction, composition of full error covariance, and state update (Mortada et al., 13 Jun 2025). Algorithmically:

Step	Formula / Description	Network Used
Prediction	$x_k \in \mathbb{R}^m$4	None
Feature Form	$x_k \in \mathbb{R}^m$5	None
Gain Update	$x_k \in \mathbb{R}^m$6	RNN #1
Covariance	$x_k \in \mathbb{R}^m$7	None
Covar Update	$x_k \in \mathbb{R}^m$8	RNN #2
Full Covar	$x_k \in \mathbb{R}^m$9	Both
Correction	$z_k \in \mathbb{R}^n$0	Both

This approach ensures that RKN’s corrected covariance estimates remain symmetric and positive semi-definite regardless of the gain network’s output, leveraging the mathematical properties of the Joseph update (Mortada et al., 13 Jun 2025).

3. Joseph’s Formula and Uncertainty Quantification

A core principle of RKN is explicit uncertainty propagation using the Joseph-formula-based update:

$z_k \in \mathbb{R}^n$1

where $z_k \in \mathbb{R}^n$2 is produced by RNN #1, and $z_k \in \mathbb{R}^n$3 is generally unknown. $z_k \in \mathbb{R}^n$4 is predicted using system dynamics. When $z_k \in \mathbb{R}^n$5 and $z_k \in \mathbb{R}^n$6 are not available, RKN learns only the $z_k \in \mathbb{R}^n$7 (noise-dependent) correction via a Cholesky parameterization, and $z_k \in \mathbb{R}^n$8 is exactly computed from known model structure and the approximated gain. This guarantees mathematically consistent error covariance estimates and distinguishes RKN from classical KalmanNet, which does not explicitly propagate error covariances (Mortada et al., 13 Jun 2025, Revach et al., 2021).

4. Training Objective and Optimization

RKN is trained end-to-end on fully observed sample trajectories $z_k \in \mathbb{R}^n$9. The estimation error at each step is $F_k \in \mathbb{R}^{m\times m}$0. The loss function is the regularized Gaussian negative log-likelihood:

$F_k \in \mathbb{R}^{m\times m}$1

Minibatch time-series training is performed with Adam optimizer, learning rate decay, and up to 200 epochs for convergence. Initial weights use Xavier/He initialization, and no prior knowledge of $F_k \in \mathbb{R}^{m\times m}$2 or $F_k \in \mathbb{R}^{m\times m}$3 is assumed or provided at any stage (Mortada et al., 13 Jun 2025).

5. Empirical Evaluation and Comparative Performance

Empirical studies in (Mortada et al., 13 Jun 2025) utilize a synthetic 1D constant-speed dynamical model with heavy-tailed, bimodal-Gaussian (non-Gaussian) measurement noise. Several baselines are considered:

• o-KF: Oracle Kalman Filter (full access to $F_k \in \mathbb{R}^{m\times m}$4)
• so-KF: Sub-optimal KF with fixed $F_k \in \mathbb{R}^{m\times m}$5
• CKN: Cholesky KalmanNet

Evaluation uses mean squared error (MSE) and mean scaled Mahalanobis distance (MSMD), measuring both prediction accuracy and consistency of covariance quantification. Across diverse noise heterogeneity levels, RKN achieves near-oracle MSE and MSMD $F_k \in \mathbb{R}^{m\times m}$6 (state dimension) for all noise regimes, outperforming both so-KF and CKN. The learned RKN gain tracks the oracle’s time-varying gain under Bernoulli mixture noise, and only RKN’s estimated covariance matches the empirical error spread (Mortada et al., 13 Jun 2025).

6. Theoretical Properties, Advantages, and Limitations

RKN’s integration of Joseph’s formula ensures that:

• All output covariances are guaranteed symmetric and positive semi-definite—critical for consistent uncertainty quantification.
• The split into $F_k \in \mathbb{R}^{m\times m}$7 (computed via model) and $F_k \in \mathbb{R}^{m\times m}$8 (learned via Cholesky network) leverages model-based information without requiring noise statistics.
• RKN demonstrates robustness to non-Gaussian, heteroscedastic, and temporally varying noise, regimes in which both classical KFs and prior variants such as Cholesky KalmanNet (CKN) and KalmanNet (Revach et al., 2021) fail to achieve consistent error quantification.

However, the generalization properties of RKN under severe out-of-distribution noise are not fully characterized; further research is needed to impose additional constraints for global statistical consistency and to extend the approach to nonlinear state-space models (e.g., via Extended or Unscented Kalman Filter assimilation) (Mortada et al., 13 Jun 2025).

7. Relationship to Related Approaches

RKN generalizes and improves upon KalmanNet (Revach et al., 2021), which replaces analytic gain computation with an RNN but does not update or propagate error covariances. While KalmanNet has demonstrated significant empirical robustness and plug-in applicability for partially known or nonlinear systems, its implicit treatment of uncertainty is limited. GSP-KalmanNet (Buchnik et al., 2023) further extends the neural-aided Kalman paradigm to graph-structured data, addressing high-dimensionality and graph topology by combining graph spectral filtering with RNN-learned gains. However, unlike RKN, these variants typically do not guarantee consistent covariance quantification, nor do they incorporate a Cholesky-based covariance network regulated by Joseph’s formula.

In summary, Recursive KalmanNet combines principled state-space recursion, mathematically consistent uncertainty propagation, and data-driven adaptation, outperforming both analytical and prior neural-augmented Kalman filtering methods in the presence of unknown and non-Gaussian noise (Mortada et al., 13 Jun 2025).