Kalman Filter: A Recursive Bayesian Estimator

• Kalman Filter is a recursive Bayesian state estimator that computes the minimum mean squared error in linear-Gaussian state-space models.
• It employs a two-step recursion—prediction and correction—to efficiently integrate noisy and incomplete measurements.
• Enhanced variants improve numerical stability and extend its application to high-dimensional, nonlinear, and robust filtering scenarios.

The Kalman Filter (KF) is a recursive Bayesian state estimator for linear dynamical systems observed via noisy and possibly incomplete measurements. First introduced by R.E. Kalman in 1960, the KF achieves minimum mean squared error (MMSE) estimation under the assumption of linear-Gaussian state-space models. Its applicability, analyzability, and computational efficiency have rendered it a foundational algorithm across control, signal processing, target tracking, econometrics, and a broad array of scientific and engineering disciplines (Yang et al., 30 Jun 2025, Benhamou, 2018, Li et al., 2014).

1. Linear-Gaussian State-Space Formulation and KF Recursion

The standard discrete-time KF operates on a system defined by

$$x_{t+1} = A x_t + w_t, \quad w_t \sim \mathcal{N}(0, W),\ y_t = C x_t + v_t, \quad v_t \sim \mathcal{N}(0, V).$$

Here, $x_t \in \mathbb{R}^n$ is an unobserved state vector, $y_t \in \mathbb{R}^p$ is the observation, $A, C$ are known system and measurement matrices, and $W, V$ are positive definite noise covariance matrices (Yang et al., 30 Jun 2025, Benhamou, 2018).

The recursive filter maintains a conditional state mean and covariance, $\hat{x}_{t|t}$ and $P_{t|t}$, via two steps:

Time-update (prediction):

$\begin{aligned} \hat{x}_{t|t-1} &= A\,\hat{x}_{t-1|t-1} \[4pt] P_{t|t-1} &= A\,P_{t-1|t-1}\,A^T + W \end{aligned}$

Measurement-update (correction):

$\begin{aligned} K_t &= P_{t|t-1}\,C^T\,(C\,P_{t|t-1}\,C^T + V)^{-1} \[4pt] \hat{x}_{t|t} &= \hat{x}_{t|t-1} + K_t\,\bigl(y_t - C\,\hat{x}_{t|t-1}\bigr) \[4pt] P_{t|t} &= (I - K_t\,C)\,P_{t|t-1} \end{aligned}$

This recursion is both finite-memory and online, with each update requiring $O(n^2p)$ floating-point operations for $n$-dimensional states and $p$-dimensional observations (Yang et al., 30 Jun 2025, Li et al., 2014).

2. Kalman Filter as the MMSE Optimal Estimator

Under the linear–Gaussian assumptions, the KF provides the (unique) MMSE estimate of the latent state given observations up to time $t$ (Yang et al., 30 Jun 2025, Benhamou, 2018). Specifically, the posterior $\mathbb{E}[x_t|y_{1:t}]$ is Gaussian, and the filter recursion updates the mean and covariance correctly at every step.

Optimality arises from the minimization of the expected squared estimation error,

$$P_{t|t} = \mathbb{E}\bigl[(x_t - \hat{x}_{t|t})(x_t - \hat{x}_{t|t})^T\,|\,y_{1:t}\bigr]$$

and the Kalman gain $K_t$ is chosen to minimize $\mathrm{tr}(P_{t|t})$ at each update (Benhamou, 2018).

3. Numerical Stability, Computational Variants, and High-Dimensional Extensions

Numerical Stability

Standard KF recursion can suffer from loss of positive definiteness or numerical instability for large or ill-conditioned systems. Factorized implementations—including Cholesky (square-root) and SVD-based algorithms—propagate matrix factors (e.g., $S_k$ such that $P_k = S_k S_k^T$) in lieu of $P_k$ itself, improving robustness. SVD-based approaches, in particular, preserve estimation accuracy in severe ill-conditioning and are algebraically equivalent to the standard KF (Kulikova et al., 2016).

High-Dimensional and Structured State Spaces

For massive-scale systems (e.g., geophysical flow imaging), direct covariance propagation is infeasible. Hierarchical matrix (e.g., $\mathcal{H}^2$) powered KFs such as "HiKF" exploit the structure and rapid off-diagonal decay of spatial covariance kernels to compress and update covariances efficiently, yielding online costs linear or near-linear in $m$ (the state dimension) (Li et al., 2014).

Space–Time Decomposition and Parallelization

Domain-decomposed KFs partition the global system along both spatial and temporal dimensions, spawning parallel local KFs that are coupled via boundary conditions and overlap corrections. This enables $O(n_{sub}^2)$ scaling with the number of subdomains and significant wall-clock speedup while remaining mathematically exact (D'Amore et al., 2023).

4. Generalizations and Nonlinear Extensions

Linear–Gaussian KFs cannot directly accommodate nonlinearities or non-Gaussian noise. Canonical extensions include:

• Extended KF (EKF): Linearizes nonlinear dynamics and observation equations about the current estimate using first-order Taylor expansions.
• Unscented KF (UKF): Utilizes sigma-point methods and the unscented transform to propagate mean and covariance through nonlinearities.
• Discriminative KF (DKF): Models the posterior $p(x|y)$ directly using a discriminative regressor (e.g., neural network, GP), providing accuracy improvements in high-dimensional observation regimes (Burkhart et al., 2016).
• Error-State and Iterated Variants (ESKF, IEKF, IESKF): Refine linearization points and parameterizations for improved performance and stability in navigation and SLAM systems (Im, 10 Jun 2024).
• Koopman KF (KKF): Lifts nonlinear dynamics via RKHS-based approximations of the Koopman operator, yielding a finite-dimensional linearized filter with $O(N^{-1/2})$ error bounds where $N$ is the basis dimension (Olguín et al., 6 Nov 2025).
• Information-Theoretic and Robust KFs: Replace the MMSE criterion with robust alternatives (e.g., maximum correntropy, error entropy (Chen et al., 2015, Chen et al., 2019), Huber loss (Yang et al., 30 Jun 2025)) to increase tolerance to outliers and heavy-tailed noise.
• Outlier-Insensitive KFs: Model potential outlier contributions as Gaussian with unknown variance (NUV) and estimate these variances online via expectation-maximization or alternating maximization (Truzman et al., 2022, Truzman et al., 2023).

A variety of robustification techniques, such as iteratively saturated correction (ISKF), are also adopted to address practical challenges in outlier-laden or adversarial environments (Yang et al., 30 Jun 2025).

5. Interpretations: Sensor Fusion, Regression, and Constrained Estimation

KF estimates can be reformulated as constrained regression or sensor-fusion problems. In particular:

• The update step is equivalent to a form of regularized least-squares regression, combining prior state prediction with new measurements under linear constraints reflecting the observation model (Jahja et al., 2019).
• Sensor fusion view: The process model can be regarded as a fictitious measurement, and the KF as fusing both process and observation "sensors" optimally.
• Linearly Constrained KF (LCKF): Allows the imposition of linear equality constraints on the filter gain, encompassing classical distortionless estimators and yielding robustness to incomplete prior information, noise mis-specification, and model uncertainty (Chaumette et al., 2017).

6. Empirical Performance and Application Domains

The KF and its variants have demonstrated high estimation accuracy and robustness in numerous settings:

• Control and Tracking: Vehicle tracking, surgical tool localization, financial time-series forecasting—attaining lower RMSE versus moving average and naive benchmarks (Ashikuzzaman et al., 2020, Benhamou, 2018).
• Sensor Fusion: Real-time fusion in GNSS/IMU, radar-lidar, and visual-inertial navigation, where the ESKF and IESKF dominate for tightly-coupled nonlinear and orientation-dominated systems (Im, 10 Jun 2024).
• Large-Scale Data Assimilation: HiKF achieves full-KF accuracy but with orders-of-magnitude faster computation and lower memory, outperforming traditional EnKF when ensemble size-limited (Li et al., 2014).
• Nonlinear System Filtering and Parameter Estimation: Koopman-KF attains the linear-Gaussian MMSE solution when applicable, and outperforms particle and extended KFs in nonlinear filtering with significantly lower computational cost (Olguín et al., 6 Nov 2025).

Comprehensive experimental and theoretical work has demonstrated the superiority of robust and adaptive variants (e.g., ISKF, OIKF, MCKF) under outliers, heavy-tailed, and multimodal noise, without sacrificing performance in ideal Gaussian environments (Yang et al., 30 Jun 2025, Truzman et al., 2022, Chen et al., 2015, Chen et al., 2019, Truzman et al., 2023).

7. Tuning, Limitations, and Best Practices

• Model Validity and Tuning: KF optimality is strictly for exact linear-Gaussian systems. In practical deployments, accuracy depends sensitively on the validity of the state-space model and the correct choice of process/measurement covariances. Empirical tuning, covariance-adaptation, and model-selection (including $\ell_1$-penalization for process models (Jahja et al., 2019)) are frequently necessary.
• Robust Filtering: Outlier-robust variants (ISKF, OIKF, MCC/MEE-KF) introduce additional parameters (e.g., saturation thresholds, kernel bandwidths). These are typically tuned via grid search or cross-validation for target RMSE or log-likelihood metrics (Yang et al., 30 Jun 2025, Truzman et al., 2022, Chen et al., 2015, Chen et al., 2019).
• Computational Considerations: Factorized implementations (e.g., SVD-KF (Kulikova et al., 2016), HiKF (Li et al., 2014)) are essential for ill-conditioned or high-dimensional deployments.
• Future Directions: Continued advances focus on nonlinear and non-Gaussian extensions, distributed and parallel filtering architectures, integration with learning-based discriminative models, and deeper theoretical characterization of robustness and stability—especially under adversarial, heavy-tailed, or model-mismatched environments (Olguín et al., 6 Nov 2025, Jiang et al., 8 Jul 2024, Singh et al., 2023).

---

References:

• "Iteratively Saturated Kalman Filtering" (Yang et al., 30 Jun 2025)
• "Trend without hiccups: a Kalman filter approach" (Benhamou, 2018)
• "A Kalman filter powered by 𝓗²-matrices for quasi-continuous data assimilation problems" (Li et al., 2014)
• "The discriminative Kalman filter for nonlinear and non-Gaussian sequential Bayesian filtering" (Burkhart et al., 2016)
• "Maximum Correntropy Kalman Filter" (Chen et al., 2015)
• "Minimum Error Entropy Kalman Filter" (Chen et al., 2019)
• "Outlier-Insensitive Kalman Filtering Using NUV Priors" (Truzman et al., 2022)
• "Outlier-Insensitive Kalman Filtering: Theory and Applications" (Truzman et al., 2023)
• "Improved Discrete-Time Kalman Filtering within Singular Value Decomposition" (Kulikova et al., 2016)
• "Koopman Kalman Filter (KKF): An asymptotically optimal nonlinear filtering algorithm with error bounds and its application to parameter estimation" (Olguín et al., 6 Nov 2025)
• "Kalman Filter, Sensor Fusion, and Constrained Regression: Equivalences and Insights" (Jahja et al., 2019)
• "Space-Time Decomposition of Kalman Filter" (D'Amore et al., 2023)
• "Linearly Constrained Kalman Filter For Linear Discrete State-Space Models" (Chaumette et al., 2017)
• "Notes on Kalman Filter (KF, EKF, ESKF, IEKF, IESKF)" (Im, 10 Jun 2024)
• "A New Framework for Nonlinear Kalman Filters" (Jiang et al., 8 Jul 2024)
• "Fast and Robust Localization of Surgical Array using Kalman Filter" (Ashikuzzaman et al., 2020)