---
title: Dynamically Iterated Filters (DIFs)
url: https://www.emergentmind.com/topics/dynamically-iterated-filters-difs
type: topic
---

# Dynamically Iterated Filters (DIFs)

Dynamically Iterated Filters (DIFs) denote a class of filtering procedures in which a nominal one-step update is replaced by an inner iteration whose update law is recomputed from the current iterate, the current data, or intermediate refined states. In the nonlinear Gaussian state-estimation literature, the term most specifically refers to a unified framework that iterates the linearization of both the transition model and the likelihood, thereby generalizing classical iterated filters such as the IEKF, IUKF, and IPLF [2404.15359]. In other literatures, closely related or explicitly named DIF constructions appear as annealing-type gain-based stochastic filters with Gaussian-sum filter banks [1304.3544], as an iterated block particle filter for coupled dynamic systems with shared and unit-specific parameters [2206.03837], and, in learned filtering, as true iterative point-cloud refinement with adaptive per-iteration supervision [2304.01529]. This suggests that the label “DIF” is used in multiple, only partially overlapping senses, while preserving a common principle: filtering by dynamically updated repeated corrections.

## 1. Scope of the term and recurring design pattern

The most direct contemporary definition comes from linearization-based nonlinear filtering. There, typical iterated filters improve the linearization point of the likelihood linearization in the EKF or UKF, but do not treat the linearization of the transition model; DIFs address both nonlinearities jointly [2404.15359]. A closely related unification frames standard filters, iterated filters, and dynamically iterated filters within a single linearization-based algorithmic skeleton [2309.07631].

Other uses of the term emphasize different mechanisms. In one strand, a DIF is a recursion that predicts an ensemble and then performs $\Gamma$ pseudo-time iterations with an additive gain-based update and an artificial diffusion parameter $\delta_\ell$ to drive the innovation process toward a zero-mean martingale [1304.3544]. In another, DIF is used for a block particle-filter extension of iterated filtering in coupled systems, with block-wise resampling, random-walk parameter perturbations, and an autoregressive correction for shared parameters [2206.03837]. In learned 3D filtering, IterativePFN is explicitly presented as instantiating a DIF by factorizing a complex filter into true per-step networks, supervising each step with adaptive targets, and allowing on-the-fly iteration control [2304.01529].

| Usage | Core mechanism | Representative paper |
|---|---|---|
| Linearization-based DIF | Iterates linearization of both $f$ and $h$ via one-step smoothing | [2404.15359] |
| Gain-based stochastic DIF | Annealing-type inner iterations with additive gain updates | [1304.3544] |
| Block-particle DIF | Block resampling with parameter perturbation and consensus correction | [2206.03837] |
| Learned iterative filtering | Repeated learned refinement with adaptive per-iteration targets | [2304.01529] |

A common misconception is to equate DIFs with any iterated filter. The more specific usage in recent nonlinear filtering is narrower: if only the measurement linearization is iterated while the prediction is kept fixed, the method reduces to a classical iterated filter rather than a DIF in the stronger sense [2404.15359].

## 2. Linearization-based DIFs for nonlinear Gaussian state-space models

The linearization-based framework assumes the discrete-time nonlinear Gaussian state-space model
$$
x_0 \sim \mathcal N(m_0,P_0),\qquad
x_k=f(x_{k-1})+q_k,\quad q_k\sim\mathcal N(0,Q_k),
$$
$$
y_k=h(x_k)+r_k,\quad r_k\sim\mathcal N(0,R_k),
$$
with hidden state $x_k\in\mathbb R^n$, measurement $y_k\in\mathbb R^m$, and possibly highly nonlinear transition and measurement functions $f$ and $h$ [2404.15359].

At each time $k$, the basic non-damped DIF solves a small joint smoothing problem over $(x_{k-1},x_k)$ by iteratively linearizing both models around the current iterates. If the $(i-1)$th iterate is denoted by $\mu_{k-1}^{(i-1)}$, $\Sigma_{k-1}^{(i-1)}$, $\mu_k^{(i-1)}$, and $\Sigma_k^{(i-1)}$, then the transition is linearized as
$$
F_k^{(i-1)}=\frac{\partial f}{\partial x}\Big|_{x=\mu_{k-1}^{(i-1)}},\qquad
d_k^{(i-1)}=f(\mu_{k-1}^{(i-1)})-F_k^{(i-1)}\mu_{k-1}^{(i-1)},
$$
and the measurement as
$$
H_k^{(i-1)}=\frac{\partial h}{\partial x}\Big|_{x=\mu_k^{(i-1)}},\qquad
e_k^{(i-1)}=h(\mu_k^{(i-1)})-H_k^{(i-1)}\mu_k^{(i-1)}.
$$
The resulting linearized two-step model is then processed by a Kalman-filter style prediction and update, followed by a Rauch–Tung–Striebel smoother step [2404.15359].

The prediction and update equations are
$$
\mu_{k|k-1}^{(i)} = F_k^{(i-1)}\mu_{k-1}^{(i-1)} + d_k^{(i-1)},
\qquad
\Sigma_{k|k-1}^{(i)} = F_k^{(i-1)}\Sigma_{k-1}^{(i-1)}(F_k^{(i-1)})^\top + Q_k,
$$
$$
S_k^{(i)} = H_k^{(i-1)}\Sigma_{k|k-1}^{(i)}(H_k^{(i-1)})^\top + R_k,
\qquad
K_k^{(i)} = \Sigma_{k|k-1}^{(i)}(H_k^{(i-1)})^\top [S_k^{(i)}]^{-1},
$$
$$
\mu_k^{(i)} = \mu_{k|k-1}^{(i)} + K_k^{(i)}\bigl[y_k - H_k^{(i-1)}\mu_{k|k-1}^{(i)} - e_k^{(i-1)}\bigr],
\qquad
\Sigma_k^{(i)} = (I-K_k^{(i)}H_k^{(i-1)})\Sigma_{k|k-1}^{(i)}.
$$
The smoothing half-step updates the previous state using
$$
G_{k-1}^{(i)}=\Sigma_{k-1}^{(i-1)}(F_k^{(i-1)})^\top[\Sigma_{k|k-1}^{(i)}]^{-1},
$$
$$
\mu_{k-1}^{(i)}=\mu_{k-1}^{(i-1)}+G_{k-1}^{(i)}\bigl[\mu_k^{(i)}-\mu_{k|k-1}^{(i)}\bigr].
$$
This structure is the defining distinction from IEKF-like procedures: the dynamic model is re-linearized and the preceding state estimate is revised inside the same inner loop [2404.15359].

The same architecture admits Jacobian-based, sigma-point, and posterior-linearization variants. Replacing Jacobians by sigma-point approximations yields the IUKF analogue; replacing them by a posterior-linearization rule yields the IPLF analogue [2404.15359]. A related formulation presents the joint variable $z=[x_{k-1};x_k]$ and the one-step cost
$$
C(z)=(x_{k-1}-\mu_{k-1|k-1})^\top P_{k-1|k-1}^{-1}(x_{k-1}-\mu_{k-1|k-1})
+(x_k-f(x_{k-1}))^\top Q_k^{-1}(x_k-f(x_{k-1}))
+(y_k-h(x_k))^\top R_k^{-1}(y_k-h(x_k)),
$$
with each DIF iteration implemented as a Gauss–Newton step solved by a two-node RTS smoother [2309.07631].

## 3. Gauss–Newton interpretation, damping, and numerical behavior

A central theoretical result is that each linearization-based DIF iteration is a Gauss–Newton step on the negative log-posterior
$$
\ell(x_{k-1},x_k)=\frac12\|x_{k-1}-m_{k-1|k-1}\|^2_{P_{k-1|k-1}^{-1}}
+\frac12\|x_k-f(x_{k-1})\|^2_{Q_k^{-1}}
+\frac12\|y_k-h(x_k)\|^2_{R_k^{-1}},
$$
or, equivalently, on a stacked residual representation $r(z)$ with $z=[x_{k-1};x_k]$ [2404.15359]. After algebra, the Gauss–Newton increment exactly recovers the Kalman-filter and RTS-smoother recursions used in the algorithm.

This interpretation enables explicit step-size correction. A damped DIF inserts a step size $\alpha^{(i)}>0$,
$$
z^{(i)} = z^{(i-1)} + \alpha^{(i)}\delta^{(i)},
$$
with $\alpha^{(i)}$ chosen, for example, by an Armijo line-search to decrease $\ell(z)$. An equivalent Levenberg–Marquardt correction adds $\lambda I$ to $J^\top J$ [2404.15359]. The significance of damping is practical rather than merely formal: the 2024 framework states that it enables explicit step-size correction and leads to damped versions of the DIFs, and reports numerically that both damped and non-damped algorithms show superior mean-squared error as well as improved parameter tuning robustness compared with analogous standard iterated filters [2404.15359].

A related unification emphasizes comparative behavior across standard, iterated, and dynamically iterated linearization-based filters. In a nonlinear localization example based on a coordinated-turn model of dimension $n=5$, twenty-five combinations of $Q$ and $R$ diagonal entries in $\{10^{-2},10^{-1},1,10,10^2\}^2$ were tested over 100 Monte Carlo runs and 50 time steps. The EKF diverged in 22/25 scenarios, whereas the Dynamically Iterated EKF remained stable in 20/25 cases; the UKF diverged in 9/25 scenarios, whereas the DI-UKF diverged in only 2/25 [2309.07631]. The same source reports average position RMSEs of **$>500$ m** for the stable EKF cases versus approximately **$12$ m** for the DI-EKF, and approximately **$35$ m** for the UKF versus approximately **$14$ m** for the DI-UKF [2309.07631]. The 2023 treatment of nonlinear transition models further reports that conventional iterated filters are not useful in a scenario with nonlinear transition model and linear measurement model, while the dynamically iterated variants have superior root mean-squared error performance and materially improved stability [2302.13871].

These results clarify a second common misconception: DIFs are not simply more iterations of an IEKF-style likelihood refinement. Their empirical advantage, as reported, is tied to re-linearization of the transition model and the one-step smoothing interpretation, not merely to repeating the measurement update [2404.15359].

## 4. Annealing-type gain-based stochastic DIFs and the filter bank

An earlier use of the term concerns nonlinear stochastic filtering for dynamic system identification. In this setting, the continuous-time state-space model is
$$
dX(t)=F(X(t),t)\,dt+G(t)\,dB(t),\qquad
Z_i=H(X(t_i))+v_i,
$$
and the target is the posterior law $p(X(t_i)\mid Z_{1:i})$ [1304.3544]. A DIF is defined as a recursion that, over each interval $[t_i,t_{i+1}]$, first predicts an ensemble by propagating the SDE and then performs $\Gamma$ pseudo-time iterations, indexed by $\ell=0,1,\dots,\Gamma$, to refine a gain-based update so as to drive the innovation process in pseudo-time $\zeta$ to a zero-mean martingale. Artificial diffusion is introduced through an annealing parameter $\delta_\ell$ to ensure sufficient exploration, or “mixing,” of the state space [1304.3544].

The inner update has the Kalman-like additive form
$$
X^{(u),i+1,\ell+1}
=
\Psi\bigl(X^{(u),i,\ell}\bigr)
+
(1+\delta_\ell)\,K^{\,i+1,\ell}\,
\bigl[Z_{i+1}-H\bigl(X^{(u),i+1,\ell}\bigr)\bigr],
$$
where $\Psi(\cdot)$ is the time-marching map, $K^{i+1,\ell}$ is recomputed from current ensemble anomalies, and $\delta_\ell\ge 0$ is the artificial diffusion parameter [1304.3544]. With
$$
S_\ell^x=[X^{(u),i+1,\ell}-\bar X^{\,i+1,\ell}]_{u=1\ldots\gamma},
\qquad
S_\ell^h=[H(X^{(u),i+1,\ell})-\overline H]_{u=1\ldots\gamma},
$$
the gain is
$$
K^{\,i+1,\ell}=S_\ell^x(S_\ell^h)^\top\bigl[S_\ell^h(S_\ell^h)^\top + R\bigr]^{-1}.
$$
A convenient annealing schedule is
$$
\delta_{\ell+1}=\alpha\,\delta_\ell,\qquad 0<\alpha<1,\qquad \delta_0=O(1),
$$
though a discontinuous drop to zero at $\ell=\Gamma$ is also permitted [1304.3544].

The method is coupled to a Gaussian-sum approximation for both prior and posterior:
$$
p_{\rm prior}(x)\approx\sum_{q=1}^{N_G}w_q^{\,i}\,\mathcal N(x;\mu_q^{\,i},\Sigma_q^{\,i}),
\qquad
p_{\rm post}(x)\approx\sum_{q=1}^{N_G}w_q^{\,i+1}\,\mathcal N(x;\mu_q^{\,i+1},\Sigma_q^{\,i+1}),
$$
with $N_G$ Gaussian components maintained in parallel as a filter bank [1304.3544]. The article explicitly cites Lemma 1 and Theorem 1, attributed to Sorensen and Alspach and to Anderson and Moore, for the statement that any smooth density can be approximated arbitrarily well in $L_1$ by a finite Gaussian sum.

The reported numerical evidence covers three test cases. In a one-dimensional growth model with $N=1000$, $N_G=10$, $\Gamma=5$, and $\delta_0=1$, the DIF Bank achieved RMSE approximately 20–50% lower than a Gaussian-sum particle filter over 100 Monte Carlo runs for both low and high measurement noise. In 2D target tracking with $N=200$, $\Gamma=10$, and $\delta_0=10$, the DIF Bank converged faster and had 30–40% lower RMSE in both $x$ and $y$ than an Auxiliary SIR filter. In shear-frame identification, the 5-DOF (20D) case with $N=400$, $N_G=10$, $\Gamma=10$, and $\delta_0=2$ showed that the DIF Bank accurately tracked all 10 parameters, while EnKF and a single DIF with $N_G=1$ showed bias and slow convergence; in the 20-DOF (80D) case with two incipient damages of 2% stiffness drop, only the DIF Bank detected the small drops in $s_{19},s_{20}$ and the corresponding damping changes [1304.3544].

The limitations are also explicit: increased per-step cost due to $\Gamma$ inner iterations and $N_G$ parallel banks, the need for off-line tuning of $\delta_0$, $\Gamma$, and $N_G$, and the absence of a fully rigorous stopping criterion for the pseudo-time iterations [1304.3544].

## 5. DIF as iterated block particle filtering in coupled dynamic systems

A different usage appears in inference for partially observed, stochastic, interacting, nonlinear dynamic processes, where each process is a “unit” and the goal is to maximize the log-likelihood
$$
\ell(\theta)=\log f_{Y_{1:N}}(y_{1:N};\theta),
$$
with parameters $\theta=(\phi,\psi_{1:U})$ comprising shared parameters $\phi$ and unit-specific parameters $\psi_u$ [2206.03837]. The latent states and observations are
$$
X_n=(X_{1,n},\dots,X_{U,n}),\qquad
Y_n=(Y_{1,n},\dots,Y_{U,n}),
$$
with Markov evolution of $X_n$, conditional independence of observations across units and time, and possible weak coupling between units [2206.03837].

Here DIF, called IBPF in the paper, extends standard iterated filtering (IF2) in three steps: it applies a Block Particle Filter to an extended model, resamples independently over a partition of units into $K$ disjoint blocks $B_1,\dots,B_K$, perturbs all parameters by a random walk, and after each block-wise resampling coalesces the block-specific copies of the shared component $\phi$ using an autoregressive correction [2206.03837]. The perturbation and cooling step is
$$
\Theta_{u,n}^{P,m,j}\sim
\mathcal N\!\Bigl(\Theta_{u,n-1}^{F,m,j},\,\sigma_n^2\cdot a^{2m/50}\Bigr),
$$
while the block weight is
$$
w_{n,j,k}^P=
\prod_{u\in B_k}
f_{Y_{u,n}\mid X_{u,n}}
\bigl(y_{u,n}\mid X_{u,n}^{P,j};\Theta_{u,n}^{P,m,j}\bigr).
$$
If $\mu_{k,n}$ denotes the block mean of the shared parameter and $\mu_n$ the overall mean, the autoregressive correction is
$$
\Phi_{B_k,n}^{F,j}\leftarrow \Phi_{B_k,n}^{F,j}+r(\mu_n-\mu_{k,n}).
$$
The heuristic motivation stated in the source is that block filtering alleviates the curse of dimensionality by resampling each block independently, while the autoregressive step prevents shared parameters from drifting apart across blocks [2206.03837].

Theoretical guarantees are asymmetric. For the special case with no shared parameters, Ning et al. (2021) prove convergence to the MLE as perturbation scale $\to 0$, number of particles $J\to\infty$, and number of iterations $M\to\infty$, under standard regularity conditions. With shared parameters and autoregressive correction, no full theoretical proof is yet available, though empirical evidence indicates continued approach to high-likelihood solutions [2206.03837].

The practical guidance is unusually detailed. Parameter transformations are recommended on log-scale for positive parameters and logit-scale for $(0,1)$ parameters. Typical initial perturbation scales are approximately 0.005, reduced to approximately 0.00125 for fine-tuning; for initial-value parameters, perturb only at $n=0$ and double $\sigma$ there. A typical cooling rate is $a=0.5$, corresponding to a 1% reduction in $\sigma$ per iteration. In the measles example, $J=4000$ was sufficient, with $J=8000$ used for final evaluation. The computational cost is stated as $O(M\cdot N\cdot J)$ simulator calls per unit per iteration; for $U=20$, $N=730$, $M=100$, and $J=4000$, one full search took approximately 24 hr on a single CPU core [2206.03837].

The 20-town measles case study used either $\theta\in\mathbb R^{260}$ or $\theta\in\mathbb R^{49}$. DIF/IBPF with $J=4000$ and $M=100$ recovered high-likelihood estimates on simulated data, improving more than 99% of random starts. On real data, however, block-wise gravity couplings did not outperform an uncoupled immigration-only model, suggesting that simple gravity coupling is not supported by those 20-town data [2206.03837]. This is a useful corrective to a third misconception: dynamic iteration does not guarantee that a more structured model is empirically favored.

## 6. Learned dynamic filtering and neural analogues

In deep learning, dynamic filtering denotes a different but related principle: filters may be generated dynamically conditioned on an input. Dynamic Filter Networks consist of a filter-generating network
$$
\Phi_\Theta:I_A\mapsto \{\mathbf f^{(i,j)}\}\subset\mathbb R^{s\times s\times c_B\times n\times d}
$$
and a dynamic filtering layer that applies those generated filters to another input $I_B$, producing an output feature map $G$ [1605.09673]. In the dynamic convolution case,
$$
G(i,j,k)=
\sum_{u=-\lfloor s/2\rfloor}^{\lfloor s/2\rfloor}
\sum_{v=-\lfloor s/2\rfloor}^{\lfloor s/2\rfloor}
\sum_{c=1}^{c_B}
\mathbf f_{u,v,c,k}\,I_B(i+u,j+v,c),
$$
where the entire filter depends on $I_A$ but is shared spatially; in dynamic local filtering, each spatial position has its own filter $\mathbf f^{(i,j)}$ [1605.09673]. Multiple such layers may be combined in a recurrent architecture, and the paper explicitly notes that multiple dynamic filter layers can be stacked or placed inside a recurrent loop.

The reported results establish the efficacy of dynamic filtering even without the later DIF terminology. On Moving MNIST video prediction, the DFN used dynamic local filtering with $9\times 9$ kernels and achieved approximately 0.64 M parameters and BCE approximately 285.2, compared with approximately 7.6 M parameters and BCE approximately 367.1 for a prior Conv-LSTM and approximately 142.7 M parameters and BCE approximately 341.2 for a prior FC-LSTM [1605.09673]. On stereo view prediction on Highway Driving, the same architecture family with $13\times 1$ horizontal filters and a dynamic per-pixel bias obtained approximately 0.46 M parameters and test loss approximately 0.52 [1605.09673]. The filters could be visualized as displacement vectors, yielding unsupervised optical-flow or disparity estimates.

IterativePFN provides a more literal learned analogue of a dynamically iterated filter. Its input is a noisy point cloud $\{\mathbf p_i^{(0)}\}_{i=1}^N$, and it applies $T$ copies of an IterationModule, each producing refined points $\{\mathbf p_i^{(t+1)}\}$ from the current cloud $\{\mathbf p_i^{(t)}\}$ [2304.01529]. Each module performs local feature extraction via a PointNet-like edge convolution, point displacement prediction, and coordinate update:
$$
\mathbf e_{ij}^{(t)}=
\mathrm{MLP}_{\mathrm{edge}}
\bigl([\mathbf p_j^{(t)}-\mathbf p_i^{(t)},\mathbf f_j^{(t)}]\bigr),\qquad
\mathbf h_i^{(t)}=\max_{j\in\mathcal N(i)}\mathbf e_{ij}^{(t)},
$$
$$
\Delta \mathbf p_i^{(t)}=\mathrm{MLP}_{\mathrm{disp}}(\mathbf h_i^{(t)}),\qquad
\mathbf p_i^{(t+1)}=\mathbf p_i^{(t)}+\Delta \mathbf p_i^{(t)}.
$$
The modules share architecture but do not share weights, which the source describes as a true iterative design that allows each step to specialize [2304.01529].

Its key training novelty is adaptive per-iteration supervision. With clean points $\{\mathbf p_i^*\}$, the target at iteration $t$ is
$$
\mathbf p_i^{*(t)}=
\mathbf p_i^{(0)}+\alpha_t(\mathbf p_i^*-\mathbf p_i^{(0)}),
\qquad
\alpha_0<\alpha_1<\cdots<\alpha_{T-1}=1,
$$
typically with $\alpha_t=\frac{t+1}{T}$, and the total loss is
$$
\mathcal L=
\sum_{t=0}^{T-1} w_t\sum_{i=1}^N
\|\mathbf p_i^{(t+1)}-\mathbf p_i^{*(t)}\|^2.
$$
All $w_t$ are set to 1 by default, though putting slightly more emphasis on early steps is also reported [2304.01529]. At test time one may choose any $T'\le T$, which provides a graceful speed–accuracy trade-off without gradients or backpropagation through time.

On the ShapeNet filtering benchmark, the reproduced results are average Chamfer $L_2\times 10^{-3}$ and normal consistency. Original noisy clouds had Chamfer 35.2 and normal consistency 0.43; PFN (single step) achieved Chamfer 12.7 and normal 0.78; IterativePFN with $T=5$ achieved **9.3** and **0.84**; a test-time ICP refine achieved 11.2 and 0.80 [2304.01529]. The source further states that real LiDAR data show similar approximately 25% gains in Chamfer over all single-step nets. A plausible implication is that, in learned filtering, “dynamic iteration” may refer less to Bayesian recursion than to the decomposition of a difficult correction into a sequence of specialized, explicitly supervised refinement stages.

Across these usages, DIFs are best understood not as a single settled algorithm, but as a family of iterative filtering constructions in which the update operator itself changes with the current iterate. In linearization-based nonlinear filtering, this means re-linearizing both dynamics and likelihood and solving a local smoothing problem [2404.15359]. In stochastic Monte Carlo filtering, it means annealed gain recomputation and Gaussian-sum filter banks [1304.3544]. In block-particle inference, it means iterated perturbation-resampling with consensus correction for shared parameters [2206.03837]. In learned systems, it means dynamically generated or stage-specific filters with adaptive intermediate targets [1605.09673; 2304.01529]. The shared technical theme is dynamic refinement; the mathematical formalism depends on the domain.

Source: https://www.emergentmind.com/topics/dynamically-iterated-filters-difs