Double State Transformation (DST) in VIO

• DST is a state-transformation framework that redefines position and velocity errors via successive, coordinate-aware transforms to restore the true four-degree-of-freedom unobservable subspace.
• It enhances filter-based visual inertial odometry by rebuilding state and measurement models, reducing RMSE by up to 33.8% and achieving 20–60% improvements over previous methods.
• Its integration in DST-EKF and DST-RTS backtracking ensures robust estimation under visual dropouts, minimizing drift and preserving consistency in the filter.

Double State Transformation (DST) is a state-transformation framework introduced within SP-VIO, a filter-based visual inertial odometry method that rebuilds the state and measurement models to improve consistency, accuracy, and robustness under visual deprived conditions (Du et al., 2024). In this formulation, the standard EKF-based VIO error-state representation is identified as computationally simple but insufficiently faithful to the true relative geometry of position and velocity errors, which leads to a spurious reduction of the unobservable subspace and thus inconsistency. DST addresses this by introducing two successive, coordinate-aware error-state transforms that redefine both velocity and position errors in the current body-frame, with the stated effect of recovering the true four-degree-of-freedom unobservable directions—global yaw and global translation—and eliminating spurious information gain.

1. Conceptual basis and motivation

Filter-based visual inertial odometry is described as having high computational efficiency and small memory requirements, which gives it a good application prospect in miniaturized and payload-constrained embedded systems (Du et al., 2024). At the same time, the filter-based method is stated to have the problem of insufficient accuracy. SP-VIO addresses this by rebuilding both the state and measurement models, and by considering further visual deprived conditions.

The specific motivation for DST is the observation that a conventional error-state EKF linearizes the IMU pose state

$$x_b=\bigl({_b^Gq},\;{}^G v_b,\;{}^G p_b,\;b_g,\;b_a\bigr)$$

using

$$\delta{}^G v_b = {}^G\tilde v_b-{}^G v_b,\qquad \delta{}^G p_b = {}^G\tilde p_b-{}^G p_b,$$

together with an additive quaternion error for orientation. In the source formulation, these definitions are characterized as “naïve” because they ignore the fact that velocity and position live in a rotating and translating global frame. The central claim is that this mismatch between the coordinate meaning of the state variables and the coordinate meaning of their additive errors is what causes the unobservable subspace to shrink spuriously under linearization.

DST is preceded by ST-EKF, which remedies the velocity error definition by expressing the velocity error in the current body-frame. DST extends that logic to position as well. This suggests that DST should be understood not as a generic refinement of linearization, but as a coordinate-consistent redefinition of the error state designed to preserve the correct observability structure.

2. Error-state formulation and the two transformations

The nominal IMU state is written as

$x_b = \bigl({_b^Gq},{}^Gv_b,{}^Gp_b,b_g,b_a\bigr),$

with perturbation

$$\delta x_b=(\,{}_b^G\phi,\;\delta{}^Gv_b,\;\delta{}^Gp_b,\;\delta b_g,\;\delta b_a).$$

DST consists of two mappings applied successively to this perturbation state. The first mapping, $f_1$, transforms the velocity error: $$\delta x_b' = f_1(\delta x_b)= \begin{bmatrix} {}_b^G\phi \ \delta_{ST}{}^Gv_b \ \delta{}^Gp_b \ \delta b_g \ \delta b_a \end{bmatrix}, \qquad \delta_{ST}{}^Gv_b = {}_b^G R\bigl({}^G\tilde v_b-{}^Gv_b\bigr) \approx \delta{}^Gv_b-[\,{}^G\tilde v_b\times]\,{}_b^G\phi.$$

The second mapping, $f_2$, transforms the position error: $$\delta x_b'' = f_2(\delta x_b')= \begin{bmatrix} {}_b^G\phi \ \delta_{ST}{}^Gv_b \ \delta_{ST}{}^Gp_b \ \delta b_g \ \delta b_a \end{bmatrix}, \qquad \delta_{ST}{}^Gp_b = {}_b^G R\bigl({}^G\tilde p_b-{}^G p_b\bigr) \approx \delta{}^Gp_b-[\,{}^G\tilde p_b\times]\,{}_b^G\phi.$$

The “double” transformation is therefore

$$f_1:\ \delta{}^Gv_b\rightarrow \delta_{ST}{}^Gv_b, \qquad f_2:\ \delta{}^Gp_b\rightarrow \delta_{ST}{}^Gp_b,$$

and the full DST is

$\delta x_b''=f_2\bigl(f_1(\delta x_b)\bigr).$

Its Jacobian with respect to the original perturbation is

$$\delta{}^G v_b = {}^G\tilde v_b-{}^G v_b,\qquad \delta{}^G p_b = {}^G\tilde p_b-{}^G p_b,$$0

where $$\delta{}^G v_b = {}^G\tilde v_b-{}^G v_b,\qquad \delta{}^G p_b = {}^G\tilde p_b-{}^G p_b,$$1 and $$\delta{}^G v_b = {}^G\tilde v_b-{}^G v_b,\qquad \delta{}^G p_b = {}^G\tilde p_b-{}^G p_b,$$2. The stated rationale is that by defining both velocity and position errors in a rotating local frame, DST guarantees that the linearized error-state Jacobians recover exactly the four true unobservable directions and thereby eliminate spurious information gain (Du et al., 2024).

3. Incorporation into DST-EKF and pose-only visual description

DST-EKF is introduced to replace the standard extended Kalman filter (Std-EKF) for improving the system's consistency. Because the error has been redefined by DST, the continuous-time error dynamics are written in the linear form

$$\delta{}^G v_b = {}^G\tilde v_b-{}^G v_b,\qquad \delta{}^G p_b = {}^G\tilde p_b-{}^G p_b,$$3

where $$\delta{}^G v_b = {}^G\tilde v_b-{}^G v_b,\qquad \delta{}^G p_b = {}^G\tilde p_b-{}^G p_b,$$4 and $$\delta{}^G v_b = {}^G\tilde v_b-{}^G v_b,\qquad \delta{}^G p_b = {}^G\tilde p_b-{}^G p_b,$$5 incorporate both rotation and gravity terms and explicitly use the transformed error definitions: $$\delta{}^G v_b = {}^G\tilde v_b-{}^G v_b,\qquad \delta{}^G p_b = {}^G\tilde p_b-{}^G p_b,$$6

After discretization over each IMU interval,

$$\delta{}^G v_b = {}^G\tilde v_b-{}^G v_b,\qquad \delta{}^G p_b = {}^G\tilde p_b-{}^G p_b,$$7

the EKF prediction step is

$$\delta{}^G v_b = {}^G\tilde v_b-{}^G v_b,\qquad \delta{}^G p_b = {}^G\tilde p_b-{}^G p_b,$$8

At update time, SP-VIO uses pose-only visual description to avoid the linearization error caused by 3D feature estimation. The visual residual is written as

$$\delta{}^G v_b = {}^G\tilde v_b-{}^G v_b,\qquad \delta{}^G p_b = {}^G\tilde p_b-{}^G p_b,$$9

and the Jacobian $x_b = \bigl({_b^Gq},{}^Gv_b,{}^Gp_b,b_g,b_a\bigr),$0 is obtained without any null-space projection. The filter update is

$x_b = \bigl({_b^Gq},{}^Gv_b,{}^Gp_b,b_g,b_a\bigr),$1

$x_b = \bigl({_b^Gq},{}^Gv_b,{}^Gp_b,b_g,b_a\bigr),$2

$x_b = \bigl({_b^Gq},{}^Gv_b,{}^Gp_b,b_g,b_a\bigr),$3

The source states that because the very definition of $x_b = \bigl({_b^Gq},{}^Gv_b,{}^Gp_b,b_g,b_a\bigr),$4 used in both propagation and update is the double-transformed one, DST-EKF fully respects the true $x_b = \bigl({_b^Gq},{}^Gv_b,{}^Gp_b,b_g,b_a\bigr),$5 d.o.f. unobservable subspace. In this sense, DST is inseparable from the filter equations in which it is embedded: it changes the meaning of the propagated and updated uncertainty, not only the notation of the state perturbation.

4. Observability, nullspace structure, and consistency

The observability analysis is central to the DST formulation (Du et al., 2024). In a partially observable VIO system, the true unobservable directions form a $x_b = \bigl({_b^Gq},{}^Gv_b,{}^Gp_b,b_g,b_a\bigr),$6 d.o.f. nullspace $x_b = \bigl({_b^Gq},{}^Gv_b,{}^Gp_b,b_g,b_a\bigr),$7, consisting of yaw plus global translation, and a consistent filter must satisfy

$x_b = \bigl({_b^Gq},{}^Gv_b,{}^Gp_b,b_g,b_a\bigr),$8

For a standard EKF, the paper states that the resulting $x_b = \bigl({_b^Gq},{}^Gv_b,{}^Gp_b,b_g,b_a\bigr),$9 depends erroneously on the estimated poses and features, so linearization will shrink it to $$\delta x_b=(\,{}_b^G\phi,\;\delta{}^Gv_b,\;\delta{}^Gp_b,\;\delta b_g,\;\delta b_a).$$0 d.o.f. ST-EKF corrects the velocity block so that $$\delta x_b=(\,{}_b^G\phi,\;\delta{}^Gv_b,\;\delta{}^Gp_b,\;\delta b_g,\;\delta b_a).$$1 no longer has spurious coupling in the second block, but still carries a dependence on the initial position. DST is then presented as yielding the simplest nullspace: $$\delta x_b=(\,{}_b^G\phi,\;\delta{}^Gv_b,\;\delta{}^Gp_b,\;\delta b_g,\;\delta b_a).$$2

This nullspace depends only on the gravity vector and the feature-to-body displacement at the current time. Because it no longer involves estimated initial states, the paper states that it remains rank-correct under all linearization, fully preserving the $$\delta x_b=(\,{}_b^G\phi,\;\delta{}^Gv_b,\;\delta{}^Gp_b,\;\delta b_g,\;\delta b_a).$$3 d.o.f. unobservable subspace. The associated interpretation in the source is explicit: DST restores observability and consistency by using two successive, coordinate-aware error-state transforms, and the comprehensive observability analysis shows that SP-VIO has a more stable unobservable subspace, which can better avoid the inconsistency problem caused by spurious information.

A common misunderstanding in discussions of filter consistency is to treat any observability-preserving modification as equivalent. The comparison among Std-EKF, ST-EKF, and DST indicates a more specific distinction. ST-EKF remedies the velocity definition only; DST also transforms the position error into the body-frame. The source therefore attributes the final recovery of the full $$\delta x_b=(\,{}_b^G\phi,\;\delta{}^Gv_b,\;\delta{}^Gp_b,\;\delta b_g,\;\delta b_a).$$4 d.o.f. unobservable subspace to the double transformation rather than to the single velocity transformation alone.

5. DST-RTS backtracking and visual interruption

SP-VIO supplements the forward DST-EKF with an enhanced double state transformation Rauch-Tung-Striebel (DST-RTS) backtracking method to optimize motion trajectories during visual interruption (Du et al., 2024). The motivation is that long visual dropouts can lead to drift that cannot be recovered by a forward filter.

The smoother operates over a block of states. Using the forward filter outputs

$$\delta x_b=(\,{}_b^G\phi,\;\delta{}^Gv_b,\;\delta{}^Gp_b,\;\delta b_g,\;\delta b_a).$$5

the smoothing gain and backward recursions are

$$\delta x_b=(\,{}_b^G\phi,\;\delta{}^Gv_b,\;\delta{}^Gp_b,\;\delta b_g,\;\delta b_a).$$6

$$\delta x_b=(\,{}_b^G\phi,\;\delta{}^Gv_b,\;\delta{}^Gp_b,\;\delta b_g,\;\delta b_a).$$7

$$\delta x_b=(\,{}_b^G\phi,\;\delta{}^Gv_b,\;\delta{}^Gp_b,\;\delta b_g,\;\delta b_a).$$8

The paper emphasizes that the backward pass uses the DST-defined covariances $$\delta x_b=(\,{}_b^G\phi,\;\delta{}^Gv_b,\;\delta{}^Gp_b,\;\delta b_g,\;\delta b_a).$$9 and thereby preserves consistency in the smoothed estimates. This suggests that the smoother is not introduced as an independent trajectory-refinement module, but as a backtracking procedure whose consistency depends on the same error-state definition used in the forward filter.

Within SP-VIO, DST-RTS is specifically associated with robustness under visual interruption and long visual dropouts. The stated role is to optimize motion trajectories when the visual stream is unavailable or degraded, complementing the forward filter, which otherwise accumulates drift.

6. Empirical evidence, comparative results, and broader applicability

Monte-Carlo simulations and real-world experiments are reported to show that SP-VIO has better accuracy and efficiency than state-of-the-art VIO algorithms, and has better robustness under visual deprived conditions (Du et al., 2024). The theoretical explanation given in the source is that by restoring the correct $f_1$0 d.o.f. nullspace in both propagation and update, DST eliminates the so-called “spurious information” that a conventional EKF or even ST-EKF would inject into yaw and translation.

The quantitative evidence summarized in the source includes two specific comparisons. First, DST-EKF yields up to a $f_1$1 reduction in RMSE on the KITTI odometry sequences over MSCKF, and comparable gains ($f_1$2–$f_1$3) over ST-EKF alone. Second, under long visual dropouts, DST-RTS reduces error growth by $f_1$4 versus $f_1$5 for VINS-Mono without smoothing. These numbers are presented as empirical support for both the consistency argument and the claimed robustness under visual interruption.

The paper also presents DST as a more general guideline for EKF-style estimation. Any EKF-style estimator whose error is defined in mixed coordinates—some parts in global frame and some in local frame—may benefit from a DST that transforms all additive errors into a single, physically meaningful local or invariant frame. The stated procedure is to identify which error components violate coordinate consistency, introduce successive transforms $f_1$6 so that each error component is re-expressed in the proper body-frame or group-invariant frame, derive the composite Jacobian

$f_1$7

and then replace the naïve $f_1$8 and $f_1$9 in propagation while computing measurement Jacobians with respect to the transformed error. The remainder of the EKF pipeline is unchanged.

The same source states that this approach can be extended to other mobile-robot and SLAM contexts, including landmark-based SLAM and multi-sensor fusion, wherever partial observability and filtering inconsistency arise from improper error definitions. A plausible implication is that DST is best understood as a coordinate-consistency principle instantiated in SP-VIO, rather than as a VIO-specific heuristic.