---
title: Tightly-Coupled Schmidt Kalman Filter
url: https://www.emergentmind.com/topics/tightly-coupled-schmidt-kalman-filter-skf
type: topic
---

# Tightly-Coupled Schmidt Kalman Filter

Searching arXiv for tightly-coupled Schmidt Kalman filter papers relevant to the provided topic.
A tightly-coupled Schmidt Kalman Filter (SKF) is a Kalman-filter variant in which the measurement model remains jointly defined over actively updated states and a set of non-updated nuisance or considered states, while the filter explicitly preserves the cross-covariances that couple the two blocks. In the tightly-coupled visual–inertial and LiDAR–inertial literature, this construction is used to incorporate map information, past poses, or other latent variables directly into the estimation equations without incurring the full computational and memory cost of a standard EKF update on all variables. In SEVIS, for example, the Schmidt-EKF VI-SLAM formulation optimally fuses IMU measurements and monocular images in a tightly-coupled manner, selectively including informative features in the state vector while treating matured features as nuisance parameters once their estimates and uncertainties have converged [1903.08636].

## 1. Definition and state partitioning

The defining structural feature of the SKF is the partition of the full state into an updated block and a frozen block. In the general discrete-time formulation, the augmented state is written as
$$
x_a = \begin{bmatrix}x \\ p\end{bmatrix},
$$
where \(x\) denotes the estimated state and \(p\) denotes considered states or parameters that are not updated directly during the measurement step [2208.12402]. In SEVIS, the full EKF state is
$$
x = [x_A; x_S],
$$
where \(x_A\) contains the IMU navigation state and a sliding window of cloned past poses, while \(x_S\) contains matured map features represented as 3D points [1903.08636]. In map-based visual–inertial localization, the same active–nuisance decomposition is applied to odometry variables and map-keyframe poses; in LODESTAR, the partition is expressed as updating and fixed blocks inside a sliding window of current and past states [2204.12108].

The associated covariance is partitioned conformally. In SEVIS,
$$
P =
\begin{bmatrix}
P_{AA} & P_{AS}\\
P_{SA} & P_{SS}
\end{bmatrix},
$$
with \(P_{AS}\) encoding the cross-correlation between active states and Schmidt states [1903.08636]. The rationale is explicit: once map features have “matured,” their means are no longer updated, but their cross-covariances with the active states continue to be maintained so that their uncertainty still informs future active-state corrections. This same logic appears in the broader SKF formulation, where the considered-state gain block is forced to zero while the corresponding covariance blocks remain part of the filter algebra [2208.12402].

A tightly-coupled SKF therefore differs from a loosely coupled architecture not by abandoning shared uncertainty, but by preserving it selectively. The frozen block is not ignored; it is held fixed in mean and, in standard SKF form, also fixed in its own covariance block, while its uncertainty continues to enter the innovation covariance and the active-state gain through cross-covariances. This suggests that “tightly coupled” refers to the measurement and covariance structure, not to whether all variables are explicitly corrected at each update.

## 2. Prediction model and partial-update equations

In tightly-coupled SKF systems, propagation generally affects only the evolving or active portion of the state. In SEVIS, the nonlinear IMU dynamics are written as
$$
x_{k+1|k} = f(x_{k|k}, u_k),
$$
and after linearization the error-state dynamics take the form
$$
\delta x_{k+1} \approx F_k\,\delta x_k + G_k\,n_k.
$$
Because the process model does not evolve Schmidt states, the transition matrix has block form
$$
F_k =
\begin{bmatrix}
\Phi_k & 0\\
0 & I
\end{bmatrix},
$$
which yields
$$
P_{AA,k+1|k} = \Phi_k P_{AA,k|k}\Phi_k^\top + \Gamma_k Q_{IMU}\Gamma_k^\top,
$$
$$
P_{AS,k+1|k} = \Phi_k P_{AS,k|k},
$$
$$
P_{SS,k+1|k} = P_{SS,k|k}.
$$
Thus the Schmidt block remains static under propagation, while the active–Schmidt correlation continues to evolve [1903.08636].

The measurement update is the point at which the SKF departs from a full EKF. For a linearized measurement
$$
y_k \approx H_x x_k + H_p p_k + v_k,
$$
the standard Schmidt update imposes
$$
K_a =
\begin{bmatrix}
K_x\\
0
\end{bmatrix},
\qquad
K_x = \bigl(P_{xx}^{-}H_x^{\top}+P_{xp}^{-}H_p^{\top}\bigr)S^{-1},
$$
with
$$
S = H_xP_{xx}^{-}H_x^{\top} + H_xP_{xp}^{-}H_p^{\top}
+ H_pP_{px}^{-}H_x^{\top} + H_pP_{pp}^{-}H_p^{\top}+R.
$$
The considered states are left unchanged in mean, but their uncertainty contributes to the residual covariance and hence to the active-state gain [2208.12402].

SEVIS instantiates the same rule for visual measurements. After linearization,
$$
r_k \approx H_A\,\delta x_A + H_S\,\delta x_S + n_{f_k},
$$
and the Schmidt-EKF modification sets \(K_S=0\) while using
$$
K_A = (P_{AA}H_A^\top + P_{AS}H_S^\top)S_k^{-1}.
$$
The state update becomes
$$
\hat x_{A,k|k} = \hat x_{A,k|k-1} + K_A r_k,
\qquad
\hat x_{S,k|k} = \hat x_{S,k|k-1},
$$
and only \(P_{AA}\) and \(P_{AS}\) are updated, while \(P_{SS}\) is frozen [1903.08636]. In LODESTAR, the same pattern is written for updating and fixed blocks, with the covariance update expressed in Joseph form so that fixed states remain unchanged but still anchor the active update via \(\mathbf{P}_{uf}\) and \(\mathbf{P}_{fu}\) [2511.09142].

## 3. Tight coupling in visual–inertial SLAM and localization

In tightly-coupled visual–inertial SLAM, measurements are modeled directly against the joint state rather than through a separate pose-estimation stage. SEVIS uses a monocular feature measurement model
$$
z_k = \pi(^{C_k}p_{f_i}) + n_{f_k},
$$
where the camera-frame feature position depends jointly on the IMU pose, extrinsics, and the global 3D feature position [1903.08636]. This direct dependence on both active navigation states and map-feature states is what makes the Schmidt construction relevant: the filter can exploit map-feature uncertainty without repeatedly updating all matured map points.

The SEVIS workflow on each camera frame is explicitly organized around multiple tightly-coupled update modes. After stochastic cloning of the current pose into the sliding window, the system tracks features, extracts descriptors, and matches to last frames and keyframes. It then performs keyframe-aided 2D-to-2D matching to identify which tracked features correspond to map features, applies an MSCKF update for features that only live in the window, applies a standard EKF update for newly initialized SLAM features, and finally applies a Schmidt update for matched map features [1903.08636]. This arrangement places Schmidt updates alongside standard EKF and MSCKF updates inside a single estimation pipeline rather than treating map matching as a separate post-processing stage.

The 2022 map-based visual–inertial localization framework extends the tightly-coupled SKF idea to an invariant-EKF setting. There, the active state is represented on a Lie group, local visual features are handled in multi-state form, and map-keyframe poses are treated as nuisance states under a Schmidt update [2204.12108]. The authors explicitly note that directly combining the invariant EKF with the Schmidt filter cannot maintain the correct observability properties of the system that considers the uncertainty of the map information, and they therefore introduce an observability-constrained technique. A plausible implication is that tightly-coupled SKF design in visual–inertial localization is not solely a computational device; it also interacts with the observability structure induced by map uncertainty.

## 4. Cross-correlation maintenance, consistency, and observability

The distinctive probabilistic role of the SKF is the preservation of cross-correlation between active and frozen variables. In SEVIS, although \(x_S\) and \(P_{SS}\) are not updated, \(P_{AS}\) and \(P_{SA}\) continue to be updated. The stated purpose is to ensure that future active updates remain consistent with the uncertainty of the frozen map features; the filter “knows” how uncertain the map is and how that uncertainty couples into the active IMU and past-pose block, even though the map’s own covariance is not re-linearized [1903.08636].

This consistency interpretation also appears in the classical map-based SKF formulation. For a state \(X=[x_R;x_M]\) with covariance partitioned as
$$
P=
\begin{pmatrix}
P_{RR}&P_{RM}\\
P_{MR}&P_{MM}
\end{pmatrix},
$$
the SKF sets the map part of the gain to zero, leaves \(x_M\) and \(P_{MM}\) unchanged, and the resulting posterior covariance satisfies
$$
P_{EKF}^+ \preceq P_{SKF}^+,
$$
so SKF remains consistent [1604.08087]. The price of this consistency in the classical form is that storing the dense map covariance \(P_{MM}\) requires \(O(n^2)\) memory for \(n\) map features [1604.08087].

The observability issue is treated in greater depth in the invariant map-based visual–inertial localization framework. There, when map-keyframe poses are uncertain, the augmented system ideally has 10 null-directions, but direct linearization breaks part of that structure in practice. The proposed remedy is to enforce
$$
H_G N = 0,
\qquad
\Phi N = N,
$$
and to replace the offending Jacobian block by an orthogonally projected form
$$
H_G^* = H_G - H_G N (N^\top N)^{-1}N^\top.
$$
Used together with the Schmidt update, this restores the correct 10-D unobservable subspace and yields what the paper describes as a fully consistent filter despite map uncertainty [2204.12108]. This suggests that a tightly-coupled SKF must often be analyzed not only as a reduced-update estimator but also as an estimator whose linearization policy determines whether the correct gauge freedoms are preserved.

## 5. Computational scaling and memory-efficient variants

The principal computational motivation for the tightly-coupled SKF is the avoidance of quadratic map updates. In SEVIS, the propagation step updates \(P_{AS}\) through multiplication by \(\Phi_k\), and \(P_{SS}\) is untouched. The measurement update forms the innovation covariance and updates \(P_{AA}\) and \(P_{AS}\) with cost \(O(n\cdot m_{match})\), where in practice the number of Schmidt features matched per image is limited to a small constant \(m_{match}\ll n\). State and window management are implemented with \(O(n)\) block copies and index swaps. The overall result is that every step scales \(O(n)\), in contrast to a full EKF-SLAM whose update would be \(O(n^2)\) because both \(P_{AS}\) and \(P_{SS}\) would need full updates [1903.08636].

A limitation of the standard SKF is that even if update cost is linear, the map covariance may remain dense and expensive to store. The Cholesky-Schmidt-Kalman filter addresses this by replacing the explicit map covariance with the sparse Cholesky factor of the map Hessian. At mapping time,
$$
H = GG^\top,
\qquad
P_{MM}=H^{-1}=(GG^\top)^{-1}=G^{-1}G^{-T},
$$
and the cross-covariance is factorized as
$$
P_{RM}=\Gamma G^{-1}.
$$
Mapped-feature updates are then performed via sparse back-solves with \(G\), and all operations cost \(O(n + \mathrm{nnz}(G))\) per feature, while memory is dominated by storing sparse \(G\), giving \(O(n)\) memory [1604.08087].

The same work introduces the submap relaxation sC-SKF, which partitions the map into overlapping submaps, each with its own Cholesky factor \(G_i\) and cross-covariance factor \(\Gamma_i\). When a feature from submap \(i\) is observed, only that submap participates in the expensive back-solve, while the remaining cross-covariance factors are updated through the active-state term. The paper states that this bounds per-update cost between linear and quadratic in the map size and remains consistent by a Schur-complement argument [1604.08087]. A plausible implication is that tightly-coupled SKF design has two separate complexity axes: update sparsity from the Schmidt partial update, and storage/computation sparsity from factorized map representations.

## 6. Adaptive and factorized extensions

Recent work extends tightly-coupled SKF ideas beyond static map-feature freezing. In LODESTAR, the Adaptive Schmidt-Kalman Filter is embedded in LiDAR–inertial odometry and uses a sliding window partitioned into active states and fixed states together with the current IMU state [2511.09142]. After each LiDAR update, the window is slid, the oldest slot is discarded, and the previous last active state is either transferred into the fixed set or removed altogether according to the condition number
$$
\chi(\mathbf{H})=\frac{\sigma_{\max}(\mathbf{H})}{\sigma_{\min}(\mathbf{H})}
$$
of its associated stacked Jacobian. If \(\chi < \mathcal{T}_\chi\), the state is deemed well-conditioned and moved into the fixed set; otherwise it is removed. The fixed states then serve as reference anchors in subsequent Schmidt updates [2511.09142].

LODESTAR also illustrates a different notion of tight coupling: current LiDAR point-to-plane residuals are augmented with residuals to past states, producing a Jacobian partition \([\mathbf{H}_u\;\mathbf{H}_f]\) that couples updating and fixed blocks directly [2511.09142]. The method further adds degeneracy-aware data exploitation, pruning weak measurements using localizability criteria and compensating by adding selected fixed-state points when the condition number remains above threshold. This suggests that adaptive SKF variants can use the active–fixed partition itself as a mechanism for degeneracy management rather than only for computational savings.

A separate extension concerns numerical robustness. The work on square-root and U–D factorized Schmidt filters formulates SKF variants that incorporate the numerical robustness of square-root and factorized filtering forms and also allow actively attempting to update the considered states [2208.12402]. In the square-root SKF, the covariance is factored as \(P_a = SS^\top\) and propagated through orthonormal transformations; in the U–D factorized SKF, \(P_a = UDU^\top\) and weighted modified Gram–Schmidt is used in propagation and update. The stated motivation is to handle nonlinearities, larger initial uncertainties, and poorly conditioned systems better than a typical Extended or Schmidt Kalman filter [2208.12402]. Within the tightly-coupled setting, such factorized forms are relevant because active–considered cross-covariance preservation can otherwise be vulnerable to numerical degradation when the augmented state is large.

## 7. Applications, reported performance, and recurring trade-offs

The tightly-coupled SKF appears in multiple application regimes: visual–inertial SLAM, map-based visual–inertial localization, map-based mobile-device localization, and LiDAR–inertial odometry. The recurring application goal is to incorporate prior map uncertainty or past-state information without fully re-estimating all nuisance variables at each update. In SEVIS, the result is described as a tightly-coupled visual–inertial SLAM that runs in real time on resource-limited platforms with \(O(n)\) complexity in the number of map features [1903.08636]. In the map-based visual–inertial localization framework, the Schmidt filter is introduced specifically so that map uncertainty can be considered while the computation cost only increases linearly with the size of the map keyframes [2204.12108].

The map-based localization literature makes the memory–accuracy–speed trade-off explicit. The C-SKF retains full SKF-consistency while replacing dense map covariance storage by a sparse Cholesky factor, and sC-SKF further reduces per-feature back-solve cost by dividing the map into overlapping submaps [1604.08087]. On a Google Tango tablet over a 60 m loop using a map built from 75 m of data, the reported position RMSE values are 6.2 cm for C-SKF, 6.6 for sC-SKF with 2 submaps, 8.3 for “perfect map” noise-inflation, and 14.7 for no map; map-uncertainty storage for the full map is approximately 76 MB for \(G\) versus approximately 1.2 GB for \(P_{MM}\); and the full pipeline runs at approximately 6 Hz [1604.08087].

The observability and consistency trade-off is emphasized in the visual–inertial localization framework based on invariant filtering. The reported comparisons across EuRoC, Kaist, 4Seasons, and YQ campus indicate NEES close to 1 for the observability-constrained Schmidt invariant filter, while variants that ignore map uncertainty or use standard EKF formulations underestimate uncertainty or diverge [2204.12108]. In LODESTAR, the adaptive SKF and degeneracy-aware data exploitation are presented as a response to ill-posed estimation in degenerate LiDAR environments, where fixed states operate as anchors and measurement selection is driven by localizability contribution and Jacobian conditioning [2511.09142].

A common misconception is that freezing nuisance states means discarding their information. The tightly-coupled SKF formulations discussed here do the opposite: they preserve nuisance-state uncertainty in the innovation covariance and maintain the cross-correlation terms needed for future active-state corrections [1903.08636]. Another common misconception is that SKF alone guarantees correct observability. The visual–inertial localization results indicate that when map uncertainty is modeled, direct linearization can still violate the correct unobservable subspace, requiring an observability-constrained technique in addition to the Schmidt update [2204.12108].

Source: https://www.emergentmind.com/topics/tightly-coupled-schmidt-kalman-filter-skf