Kalman-Takens filtering in the presence of dynamical noise

Abstract: The use of data assimilation for the merging of observed data with dynamical models is becoming standard in modern physics. If a parametric model is known, methods such as Kalman filtering have been developed for this purpose. If no model is known, a hybrid Kalman-Takens method has been recently introduced, in order to exploit the advantages of optimal filtering in a nonparametric setting. This procedure replaces the parametric model with dynamics reconstructed from delay coordinates, while using the Kalman update formulation to assimilate new observations. We find that this hybrid approach results in comparable efficiency to parametric methods in identifying underlying dynamics, even in the presence of dynamical noise. By combining the Kalman-Takens method with an adaptive filtering procedure we are able to estimate the statistics of the observational and dynamical noise. This solves a long standing problem of separating dynamical and observational noise in time series data, which is especially challenging when no dynamical model is specified.

• The paper introduces the Kalman-Takens filter, a nonparametric data assimilation method combining Takens' delay embedding and the Ensemble Kalman Filter, providing a viable alternative when explicit dynamical models are unknown.
• It presents an adaptive algorithm to estimate and separate observational and system noise covariances in the absence of a parametric model, crucial for accurate filtering.
• Numerical experiments on the Lorenz-63 system demonstrate the filter's robustness to increasing noise levels and competitive accuracy with parametric filters, especially with optimal embedding dimension choices.

The paper presents a detailed study of a nonparametric data assimilation framework that synergistically integrates Takens’ delay-coordinate reconstruction with the ensemble Kalman filter (EnKF) to yield what is termed the Kalman-Takens filter. The central contribution is to provide a viable alternative to traditional parametric filtering approaches when explicit model equations are either unavailable or unreliable, particularly in the presence of both observational and system (dynamical) noise.

The methodology replaces the known dynamical evolution operator with a local nonparametric predictor obtained from delay-coordinate embedding. Given a scalar observed time series $y_k$, the reconstruction is performed by forming delay vectors

$x_k = \left[y_k, y_{k-1}, \ldots, y_{k-d}\right],$

where $d+1$ is the embedding dimension. This reconstructed state vector is then used to forecast one time step ahead via a locally constant model,

$$\tilde{f}(x_k) = \left[ \frac{1}{N}\sum_{j=1}^{N} y_{i_j+1},\, y_k,\, \ldots,\, y_{k-d+1} \right],$$

with the forecast based on the average of the one-step evolution of the $N$ nearest neighbors in the Euclidean sense. The ensemble generated in this manner is advanced through time, and the standard Kalman update equations

$\begin{aligned} K_k &= P^{xy}_k (P^{y}_k)^{-1},\[1mm] x^{+}_k &= x^{-}_k + K_k \left(y_k - y^-_k\right),\[1mm] P^{+}_k &= P^{-}_k - K_k P^{y}_k K_k^\top, \end{aligned}$

are applied to assimilate observations and reduce the measurement noise contamination.

A significant technical challenge addressed in the paper is the estimation and separation of the system noise covariance $Q$ and observational noise covariance $R$, especially when no parametric model is available. To that end, the authors incorporate an adaptive filtering algorithm that uses the innovation sequence $\epsilon_k = y_k - y^-_k$ along with its lagged product $\epsilon_k \epsilon_{k-1}^\top$ to compute empirical estimates of these covariances. In particular, the update formulas involve using local linearizations $F_{k-1}$ (of the dynamics) and $H_{k-1}$ (of the observation function) obtained by linear regression over the ensemble data. This results in empirical estimates

$\begin{aligned} P_{k-1}^e &\approx F_{k-1}^{-1} H_k^{-1}\,\epsilon_k\,\epsilon_{k-1}^\top H_{k-1}^{-T} + K_{k-1}\,\epsilon_{k-1}\,\epsilon_{k-1}^\top H_{k-1}^{-T},\[1mm] Q_{k-1}^e &= P_{k-1}^e - F_{k-2}P^a_{k-2}F_{k-2}^\top,\[1mm] R_{k-1}^e &= \epsilon_{k-1}\epsilon_{k-1}^\top - H_{k-1}P^f_{k-1}H_{k-1}^\top, \end{aligned}$

which are then smoothed with an exponential moving average to yield stable estimates of $Q$ and $R$.

The performance of the Kalman-Takens filter is evaluated on a stochastic Lorenz-63 system wherein the dynamical equations are perturbed by Gaussian white noise processes. Numerical experiments highlight several technical points:

• Reconstruction Accuracy: For systems with moderate system noise (total variance values such as 2.4 and 15), the Kalman-Takens filter achieves root mean square errors (RMSE) that are only slightly higher than those obtained by a parametric EnKF using the exact model. For example, in one comparison with observational noise variance of 20, the parametric filter yields an RMSE of approximately 2.34, whereas the Kalman-Takens filter with 2 delays achieves an RMSE close to 2.95.
• Robustness to Noise Levels: As both the observational and dynamical noise intensities are increased, the proposed method remains robust. The adaptive estimation algorithm is able to correctly attribute and adjust for the contributions of the dynamical noise (system noise) and the observational noise. Notably, the trace of the estimated system noise covariance $\text{trace}(Q_{\text{est}})$ increases appropriately with higher noise levels, while the estimate of $R$ remains accurate around the true observational noise value.
• Dependence on Embedding Dimension: The paper discusses the trade-offs associated with the choice of the number of delays $d$. While an embedding dimension of $d+1=3$ (i.e., $d=2$) is minimally sufficient for the deterministic Lorenz attractor, increasing the embedding dimension may improve neighbor identification at low noise levels. However, in the presence of significant system noise, a higher-dimensional embedding could lead to spurious neighbor matches, and it is demonstrated that using $d=2$ often performs comparably or better than using $d=4$, particularly under high observational noise conditions.
• Forecasting Capabilities: By applying the Kalman-Takens filter to the historical training data, the observational noise in the delay-embedded space is significantly reduced, which leads to improved multistep forecasting accuracy. The filtered initial state estimates reduce the RMSE at time zero relative to the raw, unfiltered data, and the ensuing forecasts maintain a performance level close to that of the parametric approach.

In summary, the paper provides a comprehensive feasibility study demonstrating that the Kalman-Takens filter can effectively overcome the inherent challenges posed by dynamical noise in nonparametric state reconstruction and forecasting. The integration of adaptive noise covariance estimation further reinforces its practical applicability, making the method competitive with conventional parametric filters even when the underlying model structure is unknown. This work also opens several directions for further research, including a deeper investigation into the optimal selection of delay coordinates and neighborhood sizes in relation to EnKF parameters.