DKFNet: Differentiable Kalman Filter for Field Inversion and Machine Learning

Abstract: The Kalman filter is a fundamental tool for state estimation in dynamical systems. While originally developed for linear Gaussian settings, it has been extended to nonlinear problems through approaches such as the extended and unscented Kalman filters. Despite its broad use, a persistent limitation is that the underlying approximate model is fixed, which can lead to significant deviations from the true system dynamics. To address this limitation, we introduce the differentiable Kalman filter (DKF), an adjoint-based two-level optimization framework designed to reduce the mismatch between approximate and true dynamics. Within this framework, a field inversion step first uncovers the discrepancy, after which a closure model is trained to capture the discovered dynamics, allowing the filter to adapt flexibly and scale efficiently. We illustrate the capabilities of the DKF using two representative examples: a rocket dynamics model and the Allen-Cahn boundary value problem. In both cases, and across a range of noise levels, the DKF consistently reduces state reconstruction error by at least 90% compared to the classical Kalman filter, while also maintaining robust uncertainty quantification. These results demonstrate that the DKF not only improves estimation accuracy by large margins but also enhances interpretability and scalability, offering a principled pathway for combining data assimilation with modern machine learning.