---
title: 'Sky-GVIO: Urban GNSS/INS/Vision Navigation'
url: https://www.emergentmind.com/topics/sky-gvio
type: topic
---

# Sky-GVIO: Urban GNSS/INS/Vision Navigation

Searching arXiv for the relevant Sky-GVIO and closely related Sky-GVINS papers.
Sky-GVIO is a tightly coupled GNSS/INS/Vision navigation system for urban canyon environments that augments multi-sensor fusion with sky-view-image-based non-line-of-sight detection and mitigation. The formulation presented under the name “Sky-GVIO” combines GNSS observations, IMU measurements, stereo visual features, and an upward-pointing fish-eye camera whose segmented sky mask is used to classify satellites as line-of-sight or non-line-of-sight before GNSS measurements enter the estimator [2404.11070]. In the cited work, the sky segmentation module is based on a Fully Convolutional Network (FCN), the NLOS-handling method is termed S-NDM, and the fusion backbone is MSCKF-based; the system is evaluated under both SPP-related and RTK-related tightly coupled models, with the stated goal of achieving continuous and accurate positioning in severe urban canyon conditions [2404.11070]. A closely related antecedent is Sky-GVINS, which addressed the same problem family using sky segmentation for NLOS rejection in a GNSS-Visual-Inertial Navigation System rather than the FCN-based GNSS/INS/Vision formulation of Sky-GVIO [2304.04007].

## 1. Problem setting and system identity

Sky-GVIO addresses vehicle localization in urban canyons, where high-rise buildings, trees, poles, and elevated roads degrade GNSS through blockage, NLOS reception, and multipath [2404.11070]. The motivating premise is that stand-alone sensing is insufficient in such environments: GNSS is vulnerable to NLOS and multipath, INS drifts over time, and vision can degrade under low texture, dynamic scenes, lighting changes, or long-term scale and pose drift when not globally constrained [2404.11070]. The system therefore adopts multi-sensor integration, with GNSS providing absolute positioning, INS providing high-rate motion propagation, and vision providing relative pose constraints [2404.11070].

Within this fusion stack, the paper explicitly positions GNSS as the only subsystem supplying absolute position information, which makes raw GNSS quality control decisive for overall estimator performance [2404.11070]. The central claim is that tightly coupled GNSS/INS/vision integration alone is insufficient in urban canyons unless GNSS measurements are screened or reweighted using environment-aware visibility reasoning [2404.11070]. Sky-GVIO operationalizes this by using sky-view images to determine whether the sky is visible in the direction of each satellite; once the sky/non-sky segmentation is available, each satellite’s direction is projected into the fish-eye image, and the resulting visibility classification informs GNSS measurement mitigation [2404.11070].

The paper defines Sky-GVIO as an enhanced tightly coupled GNSS/INS/Vision navigation system with FCN-based sky segmentation and sky-view-image-aided NLOS detection and mitigation for urban canyons [2404.11070]. Although the acronym “GVIO” is not explicitly expanded in the manuscript, the system is described as a GNSS/INS/Vision tightly coupled estimator with odometric and navigation functionality [2404.11070]. A useful contextual distinction is provided by the related Sky-GVINS paper, in which “GVINS” denotes a GNSS-Visual-Inertial Navigation System; this suggests that Sky-GVIO belongs to the same methodological family but is formulated around GNSS/INS/Vision integration and evaluated with SPP-related and RTK-related models [2304.04007].

## 2. Architecture and sensing pipeline

Sky-GVIO takes as inputs GNSS observations, IMU measurements, stereo-camera visual measurements, and sky-view images from an upward-pointing fish-eye camera [2404.11070]. In SPP mode, the GNSS input includes pseudorange observations; in RTK-related mode, it includes double-differenced carrier-phase or RTK-related GNSS observations [2404.11070]. The system also uses satellite ephemerides, derived satellite azimuth and elevation, and offline calibration between camera and IMU frames [2404.11070].

The pipeline described in the block diagram begins with GNSS preprocessing, where satellite elevation and azimuth are computed from ephemeris and SPP or RTK observation equations are prepared [2404.11070]. In parallel, the fish-eye sky camera provides sky-view images that are semantically segmented into sky and non-sky regions by the FCN module, yielding a binary sky mask [2404.11070]. The S-NDM stage then combines the sky mask with satellite projection locations, classifies each satellite as LOS or NLOS, and adjusts the GNSS observation model or covariance accordingly [2404.11070]. IMU data undergo compensation and mechanization for state propagation, while the stereo frontend performs feature extraction, matching, and tracking [2404.11070]. The MSCKF back end then tightly fuses GNSS, INS, and vision observations to produce the navigation solution [2404.11070].

The estimator output includes position, velocity, attitude, IMU bias states, a GNSS clock state in SPP mode, double-difference ambiguity states in RTK mode, and camera clone states used by MSCKF [2404.11070]. The paper does not define a separate integrity variable, but LOS/NLOS classification acts directly as GNSS quality control through covariance adjustment [2404.11070]. The critical dependency chain is therefore from sky segmentation to LOS/NLOS detection, then to GNSS screening or reweighting, and finally to tightly coupled fusion [2404.11070].

This architectural choice differs from the earlier Sky-GVINS design chiefly in how the sky module interacts with the rest of the system. Sky-GVINS inherits a nonlinear optimization backend from GVINS and performs hard rejection of blocked satellites before factor insertion, whereas Sky-GVIO adopts an MSCKF-based tightly coupled estimator and injects sky-derived LOS/NLOS decisions into the GNSS covariance model [2304.04007; 2404.11070]. This suggests a methodological shift from binary exclusion toward mitigation by reweighting, although both systems rely on upward-looking sky visibility reasoning.

## 3. FCN-based sky segmentation

The sky segmentation module in Sky-GVIO is based on a Fully Convolutional Network, introduced because traditional methods such as thresholding, K-means, and region growing are described as sensitive to illumination and cloud conditions, especially near building edges or under elevated roads [2404.11070]. Since segmentation errors directly lead to NLOS misclassification, the paper replaces hand-crafted segmentation with a learned model [2404.11070].

The network architecture consists of a VGG19 encoder and an FCN-8s decoder [2404.11070]. The downsampling backbone includes 16 convolution layers with kernel size $3 \times 3$, stride 1, and 4 max-pooling layers [2404.11070]. The decoder uses transposed convolution for upsampling, combines 8x, 16x, and 32x feature maps, and uses skip connections to fuse multi-scale features [2404.11070]. The input is a sky-view fish-eye image of arbitrary size, and the output is a segmentation map of the same size [2404.11070]. Although the general FCN formulation allows $n+1$ channels, only two labels are needed here: sky and non-sky [2404.11070].

The paper reports an inconsistency in dataset counts. In the method section, it states that 550 training datasets were made by the authors, while the experiment section reports a training dataset of 440 images and a testing dataset of 2000 images [2404.11070]. The public dataset contains urban canyon scenes from two areas in Wuhan and includes objects such as trees, tall buildings, and light poles; manual semantic labels use blue for sky and black for non-sky [2404.11070]. The dataset is released at the stated repository URL [2404.11070].

Quantitatively, the segmentation comparison includes Otsu, K-means, Region Growing, and the proposed FCN [2404.11070]. The reported FPS and Accuracy values are as follows.

| Method | FPS | Accuracy |
|---|---:|---:|
| Kmeans | 0.34 | 49.50% |
| Otsu | 5.47 | 36.45% |
| Region growth | 3.69 | 44.96% |
| Ours | 5.96 | 97.76% |

The paper defines Accuracy as the percentage of correctly segmented images in the total processed images and argues that 5.96 FPS is sufficient for real-time use because the tightly coupled MSCKF update cycle is 1 s, matching GNSS at 1 Hz [2404.11070]. Qualitatively, the FCN is described as more robust to clouds, varying light, building boundaries, and elevated bridge scenes [2404.11070]. This contrasts with Sky-GVINS, where the preferred segmentation method was not a deep model but Otsu thresholding, selected because it provided the best empirical tradeoff on that paper’s dataset, with 96.5453% IoU at 0.0105 s per image [2304.04007]. The divergence reflects a difference in experimental conditions and segmentation criteria rather than a contradiction in the general sky-aware navigation principle.

## 4. S-NDM and satellite visibility reasoning

The paper names its sky-view-image-aided GNSS NLOS detection and mitigation method S-NDM [2404.11070]. Its operational logic is direct: a binary sky mask is computed from FCN segmentation, satellite ephemeris is used to derive azimuth and elevation, a fish-eye projection model maps each satellite into the sky-view image, and the binary sky mask at that projected pixel determines LOS or NLOS status [2404.11070]. LOS satellites are illustrated as blue dots and NLOS satellites as red dots in the manuscript [2404.11070].

A notable limitation is that Sky-GVIO does not provide the explicit fish-eye projection equations in the manuscript; it states that the projection model follows prior work [2404.11070]. Accordingly, only the conceptual mapping is available from the paper:
\[
(\text{satellite ephemeris}) \rightarrow (\text{azimuth},\text{elevation}) \rightarrow (\text{fish-eye image pixel}) \rightarrow \text{sky/non-sky test}.
\]
The calibration assumptions include offline IMU-camera extrinsic calibration, known left-right stereo calibration, and a calibrated fish-eye camera model, but the exact fisheye projection type and explicit camera-body-GNSS lever-arm treatment are not stated [2404.11070].

The mitigation stage is covariance-based rather than purely exclusion-based. The paper gives separate covariance models for LOS and NLOS pseudorange and carrier phase observations, with nominal covariance depending on elevation angle and SNR, and NLOS observations assigned larger covariance [2404.11070]. The reported formulas are garbled in formatting, but the intended structure is explicit. For carrier phase, the NLOS covariance is scaled as
\[
R^{k}_{L,\text{NLOS}} = K \times R^{k}_{L,\text{LOS}},
\]
with the adopted parameters
\[
S_1 = 50,\quad A = 30,\quad S_0 = 10,\quad a = 20,
\]
standard deviations
\[
\sigma_p = 0.3\ \text{m}, \qquad \sigma_L = 0.03\ \text{m},
\]
and scale factor
\[
K = 10
\]
[2404.11070]. The paper characterizes this as making the weighting of GNSS observations “more reasonable” [2404.11070].

This is a substantive distinction from the related Sky-GVINS approach. Sky-GVINS performs a hard visibility gate: satellites projected into non-sky regions are treated as NLOS and rejected before fusion [2304.04007]. Sky-GVIO instead supports detection and mitigation through covariance modeling and reweighting [2404.11070]. A plausible implication is that Sky-GVIO is designed to preserve more GNSS information when visibility classification is uncertain, whereas Sky-GVINS emphasizes pre-factor exclusion.

## 5. Estimation model and tightly coupled fusion

Sky-GVIO is built on MSCKF, described in the paper as an EKF-style visual-inertial fusion architecture using cloned camera poses [2404.11070]. The GNSS observation models are separated into SPP and RTK-related cases. For SPP, the raw observation equations are
\[
P = \rho + c(t_r - t^s) + I + T + \varepsilon_P \tag{1}
\]
and
\[
L = \rho + c(t_r - t^s) - I + T + \lambda N + \varepsilon_L \tag{2}
\]
[2404.11070]. The manuscript states that for SPP, equation (1) is sufficient [2404.11070]. For RTK, it uses the double-difference representation
\[
\Delta \nabla P = \Delta \nabla \rho + \Delta \nabla I + \Delta \nabla T + \Delta \nabla \varepsilon_P \tag{3}
\]
and states that double differencing eliminates or reduces satellite orbit errors, satellite clock errors, receiver clock errors, and tropospheric and ionospheric delays [2404.11070].

The INS mechanization equations in the body frame are
\[
\tilde{a}_k = a_k + b^a_k + R_n^b g^n + n_a \tag{4}
\]
and
\[
\tilde{\omega}_k = \omega_k + b^g_k + n_g \tag{5}
\]
[2404.11070]. The paper notes that, for the low-cost IMU model used, Coriolis and centrifugal forces due to Earth rotation are ignored [2404.11070]. The linearized INS error dynamics are stated as
\[
\delta \dot{p}^n = \delta v^n,
\]
\[
\delta \dot{v}^n = - R_b^n(\tilde{a}-b_a)^\wedge \delta \theta^n - R_b^n \delta b_a - R_b^n n_a,
\]
\[
\delta \dot{\theta}^n = -(\tilde{\omega}-b_g)^\wedge \delta \theta^n - \delta b_g - n_g,
\]
\[
\delta \dot{b}_a = n_{b_a}, \qquad \delta \dot{b}_g = n_{b_g} \tag{6}
\]
with 15-state INS error vector
\[
\delta x_{\text{ins}} =
\begin{bmatrix}
\delta p^n & \delta v^n & \delta \theta^n & \delta b_g & \delta b_a
\end{bmatrix}^T
\tag{7}
\]
[2404.11070].

The stereo visual measurement model is written as
\[
z^j_{c_i} =
\begin{bmatrix}
u_0 \\ v_0 \\ u_1 \\ v_1
\end{bmatrix}
=
\begin{bmatrix}
X^j_{c_0,i}/Z^j_{c_0,i} \\
Y^j_{c_0,i}/Z^j_{c_0,i} \\
X^j_{c_1,i}/Z^j_{c_1,i} \\
Y^j_{c_1,i}/Z^j_{c_1,i}
\end{bmatrix}
+\varepsilon_{\text{cam},i}
\tag{8}
\]
with left and right camera transforms given by equations (9) and (10), camera clone error state in equation (11), and visual residual in equation (12) [2404.11070].

The full error state is
\[
\delta x=
\begin{bmatrix}
\delta x_{\text{ins}}^T &
\delta x_{\text{GNSS}}^T &
\delta x_{\text{cam}}^T
\end{bmatrix}^T
\tag{13}
\]
[2404.11070]. In SPP mode, the GNSS-specific state is
\[
\delta x_{\text{GNSS,SPP}} = [\delta t_r]^T \tag{14}
\]
while in RTK mode it is
\[
\delta x_{\text{GNSS,RTK}} = [\delta \nabla \Delta N]^T \tag{15}
\]
[2404.11070]. The paper does not include troposphere or ionosphere states, GNSS extrinsic lever-arm states, or online camera-IMU extrinsic estimation in the state vector [2404.11070].

Continuous-time propagation is expressed as
\[
\dot{\delta x} = F \delta x + n \tag{16}
\]
and covariance propagation as
\[
P_{k,k-1} = \Phi_{k,k-1}\, P_{k-1}\, \Phi_{k,k-1}^T + Q_{k-1} \tag{18}
\]
with 4th-order Runge–Kutta used for discretization and propagation [2404.11070]. State augmentation for camera clones follows the standard MSCKF procedure, with the state and covariance augmented when a new image arrives and IMU-camera extrinsics \(R_o^b\) and \(p_o^b\) calibrated offline [2404.11070]. The joint tightly coupled measurement equation is
\[
\begin{bmatrix}
\delta z_P \\
\delta z_L \\
\delta z_{\text{cam}}
\end{bmatrix}
=
\begin{bmatrix}
H_P \\
H_L \\
H_{\text{cam}}
\end{bmatrix}
\delta x +
\begin{bmatrix}
\varepsilon_P \\
\varepsilon_L \\
\varepsilon_{\text{cam}}
\end{bmatrix}
\tag{21}
\]
[2404.11070].

## 6. SPP-related and RTK-related operating modes

A defining feature of Sky-GVIO is that the same sky-aided NLOS mechanism is evaluated under both SPP-related and RTK-related tightly coupled formulations [2404.11070]. In the SPP-related version, GNSS uses pseudorange observations, the GNSS-specific state includes the receiver clock offset, and NLOS handling affects pseudorange covariance and thereby the quality of global fusion [2404.11070]. The paper frames this mode as suitable for low-cost applications, smartphone navigation, and pedestrian navigation, and states that it delivers meter-level accuracy [2404.11070].

In the RTK-related version, GNSS uses double-differenced RTK observations, the state includes double-difference carrier-phase ambiguities, and carrier phase contributes stronger geometric constraints than SPP [2404.11070]. The manuscript explicitly states that “we use the RTK float solution,” so the reported results correspond to RTK float tightly coupled integration rather than an integer-fixed RTK solution [2404.11070]. This distinction is consequential because the paper does not report ambiguity fixing, integer fix rate, or fixed-solution statistics [2404.11070].

The system’s “harmonizes SPP with RTK methodologies” wording refers to this shared architecture: the same sensors, the same MSCKF backbone, the same visual subsystem, the same INS propagation, and the same S-NDM quality control are used, while only the GNSS-specific state and observation model differ between SPP and RTK [2404.11070]. In this sense, Sky-GVIO is not two separate systems but a common sky-aware fusion framework instantiated with different GNSS measurement models.

The contrast with Sky-GVINS is again instructive. Sky-GVINS extends a GVINS-style optimization framework with sky-based NLOS rejection and evaluates primarily global trajectory accuracy in open and dense urban scenarios [2304.04007]. Sky-GVIO instead emphasizes an MSCKF-based GNSS/INS/Vision formulation and explicitly compares SPP-related and RTK-related tightly coupled variants under the same sky-aware frontend [2404.11070]. This suggests a change in experimental emphasis from trajectory-level comparison against baseline GNSS-VIO systems toward mode-specific GNSS integration analysis.

## 7. Experimental results, comparative context, and limitations

The experimental vehicle platform in Sky-GVIO consists of a Septentrio mosaic-X5 mini GNSS receiver, a NovAtel GNSS-850 antenna, two forward-looking FLIR BFS-U3-31S4C-C cameras, one sky-pointing fish-eye camera FE185C057HA-1, a tactical-grade NovAtel SPAN-ISA-100C IMU, an ADIS-16470 MEMS IMU, and a time synchronization board; NovAtel ProPak7 is also mentioned in the hardware illustration context [2404.11070]. Sampling rates are GNSS at 1 Hz, MEMS-IMU at 100 Hz, forward-looking cameras at 10 Hz, and fish-eye camera at 1 Hz, with all sensors synchronized to GPS time using GNSS PPS via the time synchronization board [2404.11070]. The reference trajectory is produced by tightly coupled multi-GNSS PPK/INS bidirectional smoothing using commercial IE 8.9 software [2404.11070].

Data were collected in Wuhan on September 3, 2023 in a typical urban canyon with high-rise buildings, dense foliage, and overpasses or elevated structures [2404.11070]. The route was chosen specifically because it contains severe NLOS, multipath, cycle slips, and poor satellite geometry; the LOS satellite count in some deep urban canyon or overpass scenes dropped below 4 [2404.11070]. The experiments are real-world vehicle experiments rather than simulation [2404.11070].

For tightly coupled SPP/INS/Vision, the reported RMSE in East, North, and Up is
\[
(3.24,\ 2.14,\ 3.39)\ \text{m},
\]
whereas tightly coupled SPP/INS/Vision/Sky yields
\[
(2.07,\ 1.51,\ 2.47)\ \text{m}
\]
[2404.11070]. The relative improvements are 36% in East, 29% in North, and 27% in Up [2404.11070]. For tightly coupled RTK/INS/Vision, the reported RMSE is
\[
(0.21,\ 0.13,\ 0.36)\ \text{m},
\]
and for tightly coupled RTK/INS/Vision/Sky it is
\[
(0.16,\ 0.11,\ 0.27)\ \text{m}
\]
[2404.11070]. The relative improvements are 24% in East, 15% in North, and 25% in Up [2404.11070]. The paper summarizes these findings by stating that Sky-GVIO achieves meter-level accuracy under SPP mode and sub-decimeter precision with RTK [2404.11070].

The baselines include TC-SPP/INS/Vision, TC-RTK/INS/Vision, their sky-aware counterparts, VINS-Mono, and GVINS [2404.11070]. The paper reports GVINS RMSE as
\[
(2.50,\ 1.75,\ 2.82)\ \text{m}
\]
and notes that GVINS outperforms the authors’ baseline TC-SPP/INS/Vision because GVINS also uses Doppler measurements in addition to pseudorange, visual, and inertial constraints [2404.11070]. Sky-GVIO in SPP-related mode nevertheless outperforms GVINS, with
\[
(2.07,\ 1.51,\ 2.47)\ \text{m},
\]
which the authors attribute to stricter GNSS quality control via S-NDM [2404.11070]. VINS-Mono is said to accumulate drift significantly, and no RMSE is reported in Table III because the errors become too large [2404.11070].

The earlier Sky-GVINS results provide further comparative context. In dense urban campus scenarios, the Otsu-based Sky-GVINS variant improved ATE RMSE from 25.06 m to 4.27 m on Street02 relative to GVINS, while showing smaller gains or no penalty in more open conditions [2304.04007]. That earlier study also showed that poor segmentation quality could severely degrade performance, as seen in the local-threshold Sky-GVINS variant [2304.04007]. This reinforces a common conclusion across the two system families: sky-aware GNSS filtering is only as reliable as the sky segmentation and projection chain.

The limitations reported for Sky-GVIO are both explicit and implicit. The paper does not provide explicit sky-projection equations, uses RTK float solution only, and offers no detailed analysis of integer ambiguity fixing, fix rate, NLOS classification precision or recall, calibration sensitivity, or weather and night robustness [2404.11070]. The fish-eye camera is used only for NLOS detection rather than as a direct visual measurement source in the navigation backend [2404.11070]. The manuscript also indicates that performance may degrade if sky segmentation quality drops, fish-eye camera calibration is poor, the sky is heavily obscured, lighting changes severely, or very few LOS satellites remain [2404.11070]. Future work is stated to include using fish-eye camera data beyond NLOS detection and incorporating prior information such as high-precision maps to target centimeter-level accuracy and improve robustness [2404.11070].

Taken together, Sky-GVIO and Sky-GVINS define a coherent line of research in sky-aware GNSS-visual-inertial navigation for urban canyons. Sky-GVINS demonstrates the value of lightweight sky segmentation and hard NLOS rejection in a GVINS-based optimization framework [2304.04007]. Sky-GVIO extends the concept into an FCN-based GNSS/INS/Vision MSCKF framework with S-NDM covariance mitigation and explicit SPP-related versus RTK-related evaluation [2404.11070]. The shared methodological premise is that GNSS should not be fused as if all satellites were equally visible: the local sky geometry must first be inferred from an upward-looking sensor, and that visibility information must then regulate how GNSS enters the estimator.

Source: https://www.emergentmind.com/topics/sky-gvio