Dedicated-IO: Robust Kalman Filtering

• Dedicated-IO is a robust filtering methodology designed to counteract innovation outliers in discrete-time linear state-space models.
• The rLS.IO filter integrates a Huber-type clipping mechanism within the Kalman update to manage gross deviations in process noise effectively.
• Its optimal threshold calibration and minimax-MSE framework provide superior tracking of endogenous regime shifts compared to classical and AO-robust filters.

Dedicated-IO refers to a dedicated innovation-outlier (IO)-robust Kalman filtering methodology, specifically formulated to address gross deviations in process noise—so-called innovation outliers—in discrete-time linear state-space models. The rLS.IO filter is the canonical dedicated-IO robust Kalman filter, constructed to achieve minimax mean-squared error (MSE) optimality within contamination neighborhoods for process noise, thus ensuring rapid tracking of endogenous regime shifts in the latent state without excessive attenuation, in contrast to classical or AO (additive outlier)-robust approaches (Ruckdeschel, 2010).

1. Linear State-Space Setup and Classical Kalman Filter

The baseline is the ideal linear–Gaussian discrete-time state-space model:

• State equation:

$X_t = F_t X_{t-1} + v_t$

with $X_t \in \mathbb{R}^p$, transition matrix $F_t$, and $v_t \sim N_p(0, Q_t)$ i.i.d.

• Observation equation:

$Y_t = Z_t X_t + \varepsilon_t$

where $Y_t \in \mathbb{R}^q$, observation matrix $Z_t$, and $\varepsilon_t \sim N_q(0, V_t)$ i.i.d.

Initial conditions are $X_{0|0} = a_0$, $\Sigma_{0|0} = Q_0$, and all noises and initial state are mutually independent.

Classical Kalman recursion at each $X_t \in \mathbb{R}^p$0:

• Prediction:

$X_t \in \mathbb{R}^p$1

• Innovation:

$X_t \in \mathbb{R}^p$2

• Correction:

$X_t \in \mathbb{R}^p$3

$X_t \in \mathbb{R}^p$4

$X_t \in \mathbb{R}^p$5

2. IO-Contamination Model and Minimax Criterion

Innovation outliers are modeled as gross deviations affecting the process noise $X_t \in \mathbb{R}^p$6. Formally, for each $X_t \in \mathbb{R}^p$7, the conditional law of $X_t \in \mathbb{R}^p$8 is a mixture: with probability $X_t \in \mathbb{R}^p$9, $F_t$0 is Gaussian, and with probability $F_t$1, it follows an arbitrary distribution. The contamination neighborhood is:

$F_t$2

The robust estimation objective is minimax-MSE:

$F_t$3

3. Derivation and Algorithm of the Dedicated IO-Robust Filter (rLS.IO)

The derivation centers on the estimation of the one-step state innovation $F_t$4 from the pre-whitened innovation $F_t$5.

Under the ideal model:

• $F_t$6
• $F_t$7

Classical (non-robust) estimator:

$F_t$8

To prevent treating all large innovations as measurement noise in presence of IO-contamination, the rLS.IO filter applies Huber-type clipping to the linearized measurement noise residual:

• Residual-for-noise: $F_t$9
• Huber function: $v_t \sim N_p(0, Q_t)$0 for $v_t \sim N_p(0, Q_t)$1

The IO-robust correction is constructed as follows:

Step	Operation	Purpose
1	Compute $v_t \sim N_p(0, Q_t)$2	Classical prediction and innovation
2	$v_t \sim N_p(0, Q_t)$3	Linearization of measurement error
3	$v_t \sim N_p(0, Q_t)$4	Huber clipping of residual-for-noise
4	$v_t \sim N_p(0, Q_t)$5	Unclipped innovation attributed to state
5	$v_t \sim N_p(0, Q_t)$6	Final IO-robust state update

The resulting map is piecewise linear in $v_t \sim N_p(0, Q_t)$7, with a state-dependent correction gain.

If $v_t \sim N_p(0, Q_t)$8 is not invertible (i.e., $v_t \sim N_p(0, Q_t)$9), a suitable generalized inverse is used, or the method is embedded in a robust smoother.

4. Clipping Threshold Calibration and Implementation

The minimax optimal Huber threshold $Y_t = Z_t X_t + \varepsilon_t$0 is determined by the contamination radius $Y_t = Z_t X_t + \varepsilon_t$1 and solves:

$Y_t = Z_t X_t + \varepsilon_t$2

where $Y_t = Z_t X_t + \varepsilon_t$3, $Y_t = Z_t X_t + \varepsilon_t$4 is distributed as $Y_t = Z_t X_t + \varepsilon_t$5 under the nominal model.

Alternatively, an efficiency-tuned threshold is based on allowing a premium $Y_t = Z_t X_t + \varepsilon_t$6 on the ideal MSE:

$Y_t = Z_t X_t + \varepsilon_t$7

In practice, $Y_t = Z_t X_t + \varepsilon_t$8 (or $Y_t = Z_t X_t + \varepsilon_t$9) is precomputed offline. The online computational overhead per step is minimal, requiring (in addition to the classical Kalman steps) a matrix–vector multiplication, norm and scalar evaluation, and Huber clipping.

5. Theoretical Properties and Performance Comparison

The rLS.IO filter achieves one-step minimax-MSE optimality with respect to IO-contamination. Specifically:

• It attains the saddle point of $Y_t \in \mathbb{R}^q$0 at each step.
• Under mild regularity, this one-step argument applies recursively, making rLS.IO optimally robust to single-step IOs in time series contexts.

Comparison to alternatives:

• Classical Kalman filter: rLS.IO more effectively tracks sudden structural changes (level shifts or regime changes) because it does not absorb large innovations as measurement noise.
• AO-robust filter (rLS.AO): While rLS.AO applies Huberization to $Y_t \in \mathbb{R}^q$1, thus damping exogenous measurement outliers, rLS.IO specifically tracks endogenous state changes driven by process noise.
• Hybrid (rLS.IOAO): In environments with both IOs and AOs, rLS.AO and rLS.IO can be operated in parallel, with switch-over based on runs of large innovations, forming a hybrid filter that addresses both outlier types at the cost of a small delay.

6. Context, Significance, and Connections

Robust filtering under model misspecification is essential in applications where latent state transitions are intermittently subjected to large, unpredictable shifts. Dedicated-IO filtering, as formalized in rLS.IO, enables accurate state estimation in the presence of endogenous non-Gaussian disturbances, addressing a gap left by AO-focused robustification and the baseline Kalman approach.

The development of separate optimally robust (in the minimax sense) procedures for AO and IO regimes, their hybridization, and explicit comparison to classical ACM and repeated-median-based filters is detailed in (Ruckdeschel, 2010). A plausible implication is an expanded toolkit for robust state-space estimation in domains such as econometrics, navigation, and signal processing, where distinguishing between system-intrinsic and exogenous disruptions is critical.