Iterative Ensemble Score Filter (IEnSF)

• IEnSF is a Bayesian framework that combines score-based diffusion models with an iterative correction loop to accurately recover posterior statistics in nonlinear systems.
• It systematically reduces structural bias inherent in traditional filters by refining prior parameters and mixture weights through successive iterations.
• Applied to high-dimensional state estimation and forecasting, IEnSF improves RMSE performance and robustness under non-Gaussian conditions.

The Iterative Ensemble Score Filter (IEnSF) is a Bayesian data assimilation framework that systematically reduces error in posterior distribution estimation for high-dimensional, nonlinear dynamical systems by combining score-based diffusion models with iterative correction mechanisms. IEnSF modifies and extends the Ensemble Score Filter (EnSF), which leverages diffusion-based generative modeling and Monte Carlo score estimation to track nonlinear filtering densities without relying on Gaussianity assumptions or neural network training. By introducing an iterative refinement loop—most crucially outside the reverse-time SDE solver—IEnSF corrects structural bias and more accurately recovers posterior statistics, addressing limitations observed in non-iterative score-based filtering approaches. Applications include state and parameter estimation, long-horizon uncertainty reduction in machine-learning-driven forecasting, and geoscientific data assimilation.

1. Theoretical Foundations: Score-Based Diffusion and EnSF

IEnSF’s core framework is rooted in score-based diffusion processes, wherein the evolution of probability densities is governed by forward and reverse stochastic differential equations (SDEs). Given a state variable $X_{n+1} \in \mathbb{R}^d$ and sequential observations $Y_{n+1} \in \mathbb{R}^r$, the prediction step propagates an ensemble through the system model, generating a prior ensemble $\{x^k_{n+1|n}\}$ sampled from the dynamical model $x_{n+1}=f(x_n)+\omega_n$, with $\omega_n \sim \mathcal{N}(0, Q)$.

A synthetic pseudo-time $t \in [0, 1]$ is introduced, along which the forward SDE

$dZ_t = b(t) Z_t\,dt + \sigma(t)\,dW_t$

(typically with $b_t = -1/(1-t)$, $\sigma_t^2 = 1 + t/(1-t)$) diffuses the prior toward an isotropic Gaussian. The key computational tool is the score function,

$$S_{n+1|n}(z, t) = \nabla_z \log p_{Z_t}(z) \approx \frac{1}{K} \sum_{k=1}^K \frac{-(z - \alpha_t x^k_{n+1|n})}{\beta_t^2}\, w_k(z, t)$$

with weights $Y_{n+1} \in \mathbb{R}^r$0 determined by normalized kernel densities, and $Y_{n+1} \in \mathbb{R}^r$1, $Y_{n+1} \in \mathbb{R}^r$2.

The reverse-time SDE,

$Y_{n+1} \in \mathbb{R}^r$3

enables samples to be drawn from the target (possibly non-Gaussian) posterior density if supplied with an appropriately estimated score function.

Posterior score estimation in the classic EnSF involves augmenting the prior score with the scaled gradient of the log-likelihood,

$Y_{n+1} \in \mathbb{R}^r$4

where $Y_{n+1} \in \mathbb{R}^r$5 is a time-damping factor, commonly $Y_{n+1} \in \mathbb{R}^r$6 (Zhang et al., 23 Oct 2025, Liang et al., 20 Jan 2025, Bao et al., 2023, Tang et al., 16 Mar 2026).

2. Structural Error in Non-Iterative Score Filtering

The original EnSF suffers from two structural limitations when applied to nonlinear observation models or when the posterior deviates considerably from Gaussianity:

1. For intermediate pseudo-times $Y_{n+1} \in \mathbb{R}^r$7, the true time-dependent posterior score is not a simple sum of prior and likelihood scores; EnSF’s linear combination misses key nonlinear interactions.
2. The likelihood gradient $Y_{n+1} \in \mathbb{R}^r$8 is zero in unobserved directions, so conventional EnSF fails to update those subspaces directly (Zhang et al., 23 Oct 2025).

Theoretical analysis demonstrates that, when the prior is a Gaussian mixture and the observation model is nonlinear, the correct posterior score involves posterior mixture reweighting and the conditional expectation of the likelihood score. The EnSF formula ignores these features, introducing a persistent bias except at the endpoints $Y_{n+1} \in \mathbb{R}^r$9.

3. Iterative Ensemble Score Filter Formulation

IEnSF addresses these deficiencies by introducing an outer iterative loop that refines the score estimation and thereby the sampling of the reverse-time SDE. Each iteration consists of:

• Updating the prior parameters (mean $\{x^k_{n+1|n}\}$0, covariance $\{x^k_{n+1|n}\}$1) via smoothing from posterior ensemble realizations.
• Recomputing approximate mixture weights and the expected likelihood score at each point by evaluating at the conditional mean $\{x^k_{n+1|n}\}$2 of a reference Gaussian.
• Solving the reverse-time SDE to generate a new posterior ensemble.
• Smoothly updating the reference parameters and checking for convergence in the distributional fit.

Mathematically, given a Gaussian mixture prior and observation model $\{x^k_{n+1|n}\}$3, the IEnSF’s score update at iteration $\{x^k_{n+1|n}\}$4 is

$\{x^k_{n+1|n}\}$5

where $\{x^k_{n+1|n}\}$6 is the conditional mean using $\{x^k_{n+1|n}\}$7, with $\{x^k_{n+1|n}\}$8 a pseudo-time Jacobian and $\{x^k_{n+1|n}\}$9 the posterior weights (Zhang et al., 23 Oct 2025). The iteration continues until the difference between successive Gaussian fits falls below a threshold, typically within $x_{n+1}=f(x_n)+\omega_n$0 iterations.

Theoretical results show geometric decay in the score estimation error,

$x_{n+1}=f(x_n)+\omega_n$1

with contraction factor $x_{n+1}=f(x_n)+\omega_n$2 depending on both the observation nonlinearity and the prior-posterior KL divergence (Zhang et al., 23 Oct 2025).

4. Algorithmic Workflow and Implementation

A standard IEnSF assimilation step proceeds as:

1. Forward propagate the prior ensemble via the (possibly learned) dynamical model.
2. Estimate the prior score via mini-batch Monte Carlo from the ensemble.
3. Incorporate observation likelihood gradient, with damping schedule for pseudo-time.
4. Iteratively refine the posterior score function and generate the ensemble posterior via reverse-time SDE simulations.
5. Optionally, smooth parameter updates (for joint state-parameter estimation) using a direct Bayesian filter, resampling particles according to likelihood weights (Bao et al., 2023).
6. For partial observation scenarios, post-process unobserved state components using image inpainting or learned dictionary approaches (Liang et al., 20 Jan 2025).

Key hyperparameters include the ensemble size, minibatch size, number of reverse-time SDE steps (typically $x_{n+1}=f(x_n)+\omega_n$3-$x_{n+1}=f(x_n)+\omega_n$4), smoothing factors, and iteration cap $x_{n+1}=f(x_n)+\omega_n$5 (Tang et al., 16 Mar 2026, Zhang et al., 23 Oct 2025). No neural network-based score model training is required, enhancing scalability.

5. Applications and Benchmark Results

IEnSF demonstrates broad utility in:

High-dimensional state estimation: For Lorenz-96 (1000D), IEnSF matches LETKF RMSE under linear observations and outperforms by 20–30% under nonlinear (e.g., $x_{n+1}=f(x_n)+\omega_n$6) observations (Zhang et al., 23 Oct 2025). In 200D Lorenz-96 state/parameter tracking, IEnSF achieves state RMSE $x_{n+1}=f(x_n)+\omega_n$7 versus $x_{n+1}=f(x_n)+\omega_n$8 for AugEnKF and up to $x_{n+1}=f(x_n)+\omega_n$9 reduction in parameter RMSE (Bao et al., 2023).

Data-driven forecasting: In hybrid ML-dynamical modeling (e.g., using LSTM or R-DeepONet surrogates), IEnSF stabilizes long-term prediction and reduces forecast uncertainty. On Lorenz-96, IEnSF lowers RMSE by 30–40% over EnKF, while in KdV equation experiments, it achieves a 35–50% error reduction over EnKF or unfiltered ML (Tang et al., 16 Mar 2026).

Partial/incomplete observations: Incorporating inpainting for unobserved states allows IEnSF variants to handle geophysical assimilation with sparse or indirect measurements, performing robustly under both linear and nonlinear observational operators (Liang et al., 20 Jan 2025).

6. Comparative Performance and Limitations

IEnSF offers distinct advantages over ensemble Kalman filters and particle filter approaches:

• Non-Gaussianity: No requirement for linear-Gaussian assumptions; score-based sampling captures nonlinear, multi-modal posteriors directly.
• High-dimensionality stability: Avoids particle filter weight collapse; posterior shape is encoded through score functions rather than sample covariance.
• Reduced bias via iteration: The outer iterative loop eliminates the structural error intrinsic to single-pass EnSF and enables accurate posterior recovery even in strongly nonlinear regimes.

Performance is competitive in wall-clock time with EnKF when hardware-parallelized, with robustness observed for $\omega_n \sim \mathcal{N}(0, Q)$0 and under strong model errors. Tuning is required for ensemble, minibatch, and SDE discretization parameters, with typical guidance favoring small smoothing to avoid oscillations and careful variance splitting to balance non-Gaussian support and filter stability (Bao et al., 2023, Zhang et al., 23 Oct 2025, Tang et al., 16 Mar 2026).

7. Practical Guidance and Extensions

Best practices in implementing IEnSF include:

• Ensemble size $\omega_n \sim \mathcal{N}(0, Q)$1–$\omega_n \sim \mathcal{N}(0, Q)$2, minibatch size $\omega_n \sim \mathcal{N}(0, Q)$3–$\omega_n \sim \mathcal{N}(0, Q)$4 for practical score estimation.
• Reverse SDE integration with $\omega_n \sim \mathcal{N}(0, Q)$5, pseudo-time discretized with $\omega_n \sim \mathcal{N}(0, Q)$6–$\omega_n \sim \mathcal{N}(0, Q)$7 steps.
• Early stopping in the outer iteration when successive Gaussian approximations match to within $\omega_n \sim \mathcal{N}(0, Q)$8-divergence $\omega_n \sim \mathcal{N}(0, Q)$9.
• Covariance localization and inflation are recommended for very high-dimensional systems.
• For computationally burdensome nonlinear observation operators, zeroth-order weight approximations are permissible (Zhang et al., 23 Oct 2025).
• Parallelization is natural as ensemble member updates are independent.

Typical maximum outer-loop iterations are $t \in [0, 1]$0–$t \in [0, 1]$1, with convergence achieved rapidly even in strongly nonlinear, high-dimensional applications (Zhang et al., 23 Oct 2025, Tang et al., 16 Mar 2026).

The IEnSF formulation integrates seamlessly with joint state-parameter filtering via direct Bayesian updates and, through inpainting modules, extends to partial observation and missing data scenarios. These attributes make IEnSF a versatile tool for modern data assimilation problems where classical Gaussian or sample-based methods exhibit prohibitive bias or instability.