Ensemble Kalman Inversion (EnKI)

• Ensemble Kalman Inversion (EnKI) is a derivative-free method that iteratively updates an ensemble of parameter estimates using empirical covariances.
• The algorithm strategically applies both deterministic and stochastic updates with localization to overcome subspace limitations in high-dimensional settings.
• Its localized variant (LEKI) ensures ensemble collapse and sublinear global convergence, enhancing performance for complex inverse problems.

Ensemble Kalman Inversion Algorithm (EnKI) is a derivative-free, ensemble-based method for the numerical solution of inverse problems in finite- and infinite-dimensional settings. It solves weighted least-squares optimization tasks by iteratively updating an ensemble of parameter estimates using empirical covariance structures derived from the ensemble itself. EnKI is notable for its subspace property, deterministic and stochastic variants, strong theoretical convergence guarantees, and extensions such as localization, regularization, and algorithmic acceleration. This article details its mathematical foundation, algorithmic workflow, theoretical principles, localization strategies, convergence results, and practical applications, drawing on current research—including "Localization in Ensemble Kalman inversion" (Tong et al., 2022) and related works.

1. Mathematical Foundations and Ensemble Update Formulation

EnKI targets inverse problems expressed as the recovery of an unknown $u \in \mathbb{R}^{d_u}$ from noisy observations $y \in \mathbb{R}^{d_y}$ of a forward model $G: \mathbb{R}^{d_u} \to \mathbb{R}^{d_y}$: $$y = G(u) + \eta, \quad \eta \sim \mathcal{N}(0, I_{d_y})$$ The estimation task is typically recast as minimization of the data misfit functional: $l(u) = \|G(u) - y\|^2$ Or, when imposing a Gaussian prior $\mathcal{N}(0, C_0)$,

$$l_{\text{Tik}}(u) = \|G(u) - y\|^2 + \|u\|^2_{C_0^{-1}}$$

An ensemble $\{u^j(n)\}_{j=1}^J \subset \mathbb{R}^{d_u}$ is propagated across iterations $n$; empirical mean and covariances at iteration $n$ are: $$\bar{u}(n) = \frac{1}{J} \sum_{j=1}^J u^j(n), \quad G(n) = \frac{1}{J} \sum_{j=1}^J G(u^j(n))$$

$$C^{uu}(n) = \frac{1}{J-1} \sum_j [u^j(n) - \bar{u}(n)][u^j(n) - \bar{u}(n)]^\top$$

$$C^{up}(n) = \frac{1}{J-1} \sum_j [u^j(n) - \bar{u}(n)][G(u^j(n)) - G(n)]^\top$$

$$C^{pp}(n) = \frac{1}{J-1} \sum_j [G(u^j(n)) - G(n)][G(u^j(n)) - G(n)]^\top$$

The standard (non-localized) EnKI update is: $$u^j(n+1) = u^j(n) + C^{up}(n)[C^{pp}(n) + I]^{-1}[y - G(u^j(n))]$$ Additive inflation (optional) is achieved by

$+ \lambda_n \xi^j(n)$

where $\xi^j$ is a zero-mean noise boosting $C^{uu}$.

2. Subspace Property, Limitations, and Motivation for Localization

A defining property of standard EnKI is its "subspace property": the iterative updates for ensemble members are confined to the linear span $S_0 = \mathrm{span}\{u^j(0)\}$ of the initial ensemble. Thus, unless $J \geq d_u + 1$, or $S_0$ is chosen to be sufficiently expressive, EnKI cannot generically reach the true solution outside the initial span. This constraint can severely limit performance in high-dimensional settings, creating the need for strategies that break the subspace restriction without demanding unmanageably large ensembles (Tong et al., 2022).

3. Covariance Localization: Theory and Implementation

To escape the subspace restriction and improve performance in high-dimensional or spatially-structured problems, covariance localization is employed. This process enforces an assumed spatial correlation structure on the empirical covariances. Define a localization matrix $\Psi \in \mathbb{R}^{d_u \times d_u}$, symmetric and positive semidefinite, with diagonal $\Psi_{ii}=1$, and rapid decay off-diagonal ($\Psi_{ij} \approx 0$ for $|i-j| \gg 1$).

The localized parameter covariance is constructed as: $C^{uu,\text{loc}} = C^{uu} \circ \Psi$ where $\circ$ indicates the Schur/Hadamard (elementwise) product.

Localizing $C^{up}$ commonly follows two schemes:

• Linear (linearized) localization: For approximately linear $G(u)$, define $C^{up,\text{loc}} = [ C^{uu} \circ \Psi ] H^\top$, $H \approx \nabla G$.
• Central (observation-wise) localization: For spatially-local observations $G_j(u)$, use $$(C^{up,\text{loc}})_{ij} = C^{up}_{ij} \cdot \psi( \mathrm{dist}(i, i(j)) / R_l )$$ with taper function $\psi$.

Localized EnKI (LEKI) then substitutes $C^{up,\text{loc}}$ for $C^{up}$ in the update: $$u^j(n+1) = u^j(n) + C^{up,\text{loc}}(n)[C^{pp}(n)+I]^{-1}[y-G(u^j(n))] + \lambda_n \xi^j(n)$$

4. Ensemble Collapse and Global Convergence with Localization

Under regularity and observability assumptions, the LEKI ensemble exhibits "ensemble collapse" while maintaining full rank in the localized subspace. The evolution of ensemble covariance $V(t) = C^{uu}(t)$ is governed by: $$\frac{dV}{dt} = - C^{up,\text{loc}}C^{pu,\text{loc}} - C^{up,\text{loc}}C^{pu,\text{loc}} + \lambda(t)\Sigma$$ where $\Sigma$ is the inflation covariance. Under proper conditions, for all $t \ge t_0$,

$$\frac{m}{1+t} \leq \lambda_{\min}(V(t)) \leq \lambda_{\max}(V(t)) \leq \frac{M}{1+t}$$

This ensures asymptotic collapse at rate $O(1/t)$ but preserves rank in the enriched subspace.

Moreover, under strong convexity of $l(u)$ and mild regularity of $G$, the LEKI mean converges globally to the optimal solution with sublinear rate: $$\ell(t)=l(\bar{u}(t))-l(u^*) = O((1+t)^{-c_\psi}) + O( d_y (1+t)^{-1+\epsilon} \log(1+t) )$$ If $c_\psi > 1$, the rate is $O(1/t)$. Uniform decay applies to local misfits when $G(u)$ decomposes into localized convex components (Tong et al., 2022).

5. Comparison of Non-localized and Localized EKI: Practical and Numerical Aspects

A head-to-head algorithmic comparison underscores the practical impact of localization:

• Vanilla EnKI:

1. Compute ensemble mean/covariance.
2. Update using non-localized cross-covariance.
3. Add inflation (optional).

• LEKI:
  • $C^{uu} \gets C^{uu}\circ\Psi$
  • $C^{up} \gets (C^{uu}\circ\Psi) H^\top$ or $C^{up}\circ\Psi_{\text{cross}}$.
  • 3. Update as above.

Numerical experiments confirm that LEKI breaks the initial subspace constraint, achieves much lower ensemble spread, and robustly drives the mean toward a minimizer even with $J \ll d_u$. Notable cases include linear independent-component estimation, nonlinear averaging models, Lorenz'96 dynamical inversion, and field data-driven resistivity inversion. LEKI persistently outperforms standard EnKI at moderate ensemble sizes and sustains stability in the presence of moderate model non-idealities (Tong et al., 2022).

6. Extensions, Theory-Guided Best Practices, and Robustness

Applying localization to EnKI mandates careful selection of the localization matrix and tapering function, informed by the assumed spatial structure of $G$ and the observational operator. Required theoretical conditions include Lipschitz continuity, full observability under localization, and tapering suited to the measurement/parameter topology. While proofs rely on idealized regularity, numerical evidence attests to robust performance under approximate validity of these conditions.

Overall, localized ensemble Kalman inversion (LEKI) constitutes a theoretically-grounded, computationally-efficient mechanism for overcoming EnKI's subspace limitations, equipping practitioners to tackle high-dimensional, spatially-structured inverse problems with rigorous guarantees on ensemble collapse and global convergence (Tong et al., 2022).