Hybrid Qf-ES-EKF/UKF: Efficient VIO Fusion

• The paper introduces a hybrid sensor fusion filter that integrates ESKF for full state propagation with a selective SUKF for quaternion-based orientation refinement.
• This method achieves high orientation accuracy while reducing computational cost by over 10× compared to a full SUKF, making it ideal for real-time UAV applications.
• Adaptive visual measurement covariance further enhances robustness by dynamically tuning noise models based on real-time image quality metrics, significantly improving rotation and position estimates.

A Hybrid Quaternion-focused Error-State EKF/UKF (Qf-ES-EKF/UKF) is a sequential sensor fusion technique that combines the computational efficiency of the Error-State Extended Kalman Filter (ESKF) for global state propagation with the high-fidelity nonlinear estimation of a Scaled Unscented Kalman Filter (SUKF) applied selectively to a rotation-critical subspace, specifically the orientation quaternion. The architecture was introduced for robust, adaptive visual-inertial odometry (VIO) in Unmanned Aerial Vehicles (UAVs), with mechanisms for dynamic sensor reliability assessment and measurement noise adaptation to enhance pose estimation under challenging conditions (Asil et al., 19 Dec 2025).

1. State and Error-State Representations

The Qf-ES-EKF/UKF framework utilizes a 16-dimensional nominal state vector:

• Quaternion $q \in \mathrm{SO}(3)$ (4 × 1, body-to-world orientation)
• Velocity $v \in \mathbb{R}^3$ (world frame)
• Position $p \in \mathbb{R}^3$ (world frame)
• Accelerometer bias $b_a \in \mathbb{R}^3$
• Gyroscope bias $b_g \in \mathbb{R}^3$

In vector form:

$$x = [q^\top, v^\top, p^\top, b_a^\top, b_g^\top]^\top, \qquad \dim(x) = 16$$

The associated 15-dimensional error-state vector is:

• Attitude error $\theta \in \mathbb{R}^3$ (so(3) vector)
• Velocity error $\delta v \in \mathbb{R}^3$
• Position error $\delta p \in \mathbb{R}^3$
• Accelerometer bias error $\delta b_a \in \mathbb{R}^3$
• Gyroscope bias error $v \in \mathbb{R}^3$0

With

$v \in \mathbb{R}^3$1

State correction and composition employ the “ ⊕ ” operator for quaternion error injection:

• $v \in \mathbb{R}^3$2
• $v \in \mathbb{R}^3$3

2. Error-State EKF Propagation

The core propagation leverages ESKF for the full state. The continuous-time nominal dynamics are:

• $v \in \mathbb{R}^3$4
• $v \in \mathbb{R}^3$5
• $v \in \mathbb{R}^3$6
• $v \in \mathbb{R}^3$7
• $v \in \mathbb{R}^3$8

Here, $v \in \mathbb{R}^3$9 is the quaternion multiplication matrix, and $p \in \mathbb{R}^3$0 is the rotation matrix from $p \in \mathbb{R}^3$1.

The 15-dimensional error-state is propagated with a linear time-varying system:

$p \in \mathbb{R}^3$2

Discretization uses the Van-Loan method on a $p \in \mathbb{R}^3$3 block matrix to compute the state-transition matrix $p \in \mathbb{R}^3$4 and discrete process noise $p \in \mathbb{R}^3$5.

The ESKF propagation step at each IMU sample is:

• $p \in \mathbb{R}^3$6
• $p \in \mathbb{R}^3$7

3. Quaternion-Focused SUKF Orientation Refinement

After the global ESKF propagation, a targeted SUKF refinement step is applied solely to the $p \in \mathbb{R}^3$8 orientation-error covariance block $p \in \mathbb{R}^3$9. The process is as follows:

• Generate $b_a \in \mathbb{R}^3$0 sigma points in the orientation subspace ($b_a \in \mathbb{R}^3$1) using scaled unscented transform parameters $b_a \in \mathbb{R}^3$2.
• Compute Cholesky decomposition $b_a \in \mathbb{R}^3$3, define $b_a \in \mathbb{R}^3$4, and construct $b_a \in \mathbb{R}^3$5.
• Map each sigma point to quaternion perturbations via $b_a \in \mathbb{R}^3$6 and apply to the nominal quaternion.
• Each perturbed quaternion is propagated through IMU integration for orientation only.
• Retract propagated quaternions to the nominal orientation using the log map, yielding error vectors $b_a \in \mathbb{R}^3$7.
• Refine $b_a \in \mathbb{R}^3$8 using the sigma-point statistics:

$b_a \in \mathbb{R}^3$9

• Replace original $b_g \in \mathbb{R}^3$0 in the error covariance with $b_g \in \mathbb{R}^3$1, leaving other submatrices unchanged.

4. Sequential Hybrid Propagation and Update

The complete Qf-ES-EKF/UKF filter loop includes:

1. ESKF predict for full nominal state and error covariance.
2. Extract $b_g \in \mathbb{R}^3$2 and refine with SUKF in the quaternion subspace.
3. When a visual or zero-velocity measurement becomes available, update the full error state with the standard EKF measurement update using measurement Jacobian $b_g \in \mathbb{R}^3$3 and noise $b_g \in \mathbb{R}^3$4.
4. Inject estimated error $b_g \in \mathbb{R}^3$5 into the nominal state and reset the error state.

This sequence achieves computational savings by limiting the costly SUKF step to a three-dimensional subspace while maintaining full ESKF update for all state variables.

5. Adaptive Visual Measurement Covariance

Measurement noise adaptation is governed by a dynamic sensor confidence score derived from real-time image quality metrics, including:

• Inverted (normalized) entropy, normalized blur, pose-chi$b_g \in \mathbb{R}^3$6 error, culled keyframes ratio for position covariance ($b_g \in \mathbb{R}^3$7)
• Derivative metrics (intensity, blur, chi$b_g \in \mathbb{R}^3$8, keyframes) for velocity covariance ($b_g \in \mathbb{R}^3$9)

A combined utility measure ($$x = [q^\top, v^\top, p^\top, b_a^\top, b_g^\top]^\top, \qquad \dim(x) = 16$$0 or $$x = [q^\top, v^\top, p^\top, b_a^\top, b_g^\top]^\top, \qquad \dim(x) = 16$$1) is calculated as the maximum among individual metrics (static or dynamic), transformed through a CASEF activation function:

$$x = [q^\top, v^\top, p^\top, b_a^\top, b_g^\top]^\top, \qquad \dim(x) = 16$$2

Covariance values are selected using a thresholding and interpolation scheme based on confidence scores, ensuring higher noise is assigned to outlier or low-confidence measurements.

The visual measurement covariance in the EKF update is then:

$$x = [q^\top, v^\top, p^\top, b_a^\top, b_g^\top]^\top, \qquad \dim(x) = 16$$3

6. Computational Complexity

The computational complexity is as follows:

Method	Complexity	Steps Included
Full ESKF	$$x = [q^\top, v^\top, p^\top, b_a^\top, b_g^\top]^\top, \qquad \dim(x) = 16$$4	Full state prediction
Full SUKF (16D)	$$x = [q^\top, v^\top, p^\top, b_a^\top, b_g^\top]^\top, \qquad \dim(x) = 16$$5	Sigma point expansion
Qf-ES-EKF/UKF (hybrid, 3D UKF)	$$x = [q^\top, v^\top, p^\top, b_a^\top, b_g^\top]^\top, \qquad \dim(x) = 16$$6	ESKF + 3D SUKF

With $$x = [q^\top, v^\top, p^\top, b_a^\top, b_g^\top]^\top, \qquad \dim(x) = 16$$7, $$x = [q^\top, v^\top, p^\top, b_a^\top, b_g^\top]^\top, \qquad \dim(x) = 16$$8, the hybrid yields $$x = [q^\top, v^\top, p^\top, b_a^\top, b_g^\top]^\top, \qquad \dim(x) = 16$$9252 operations vs. $\theta \in \mathbb{R}^3$03375 for full SUKF—a more than $\theta \in \mathbb{R}^3$1 reduction in state-propagation step. This enables real-time VIO on UAVs where high-fidelity orientation estimation is critical, without the prohibitive cost of applying a full SUKF to the complete state.

7. Experimental Results and Performance

Evaluation was performed on the EuRoC MAV (MH01–MH05) dataset:

• Rotation accuracy (quaternion RMSE, degrees, avg. over all sequences):
  • ESKF: 1.42°
  • SUKF: 0.85° $\theta \in \mathbb{R}^3$2
  • Qf-ES-EKF/UKF (no adaptive covariance): 0.74° $\theta \in \mathbb{R}^3$3
  • Adaptive Qf-ES-EKF/UKF: 0.61° $\theta \in \mathbb{R}^3$4
• Position accuracy (ATE, meters; MH04 + MH05):
  • ESKF: 0.56 m
  • SUKF: 0.56 m
  • Qf-ES-EKF/UKF: 0.53 m
  • Adaptive Qf-ES-EKF/UKF: 0.26 m $\theta \in \mathbb{R}^3$5
• Timing (filter only, CPU, s):
  • ESKF: 2.44 s (RTF$\theta \in \mathbb{R}^3$60.92)
  • SUKF: 147.2 s (RTF$\theta \in \mathbb{R}^3$70.092)
  • Qf-ES-EKF/UKF: 76.9 s (RTF$\theta \in \mathbb{R}^3$80.175)

The hybrid approach achieves SUKF-level orientation accuracy at approximately 48% lower computational cost compared to a full SUKF, and only %%%%69$b_a \in \mathbb{R}^3$070%%%% slower than plain ESKF. This attests to its suitability for real-time UAV VIO, especially in scenarios that are rotation-critical and subject to variable sensor reliability (Asil et al., 19 Dec 2025).