SkySplat: RPC-Native Satellite 3D Reconstruction
- SkySplat is a self-supervised framework that reconstructs 3D satellite scenes by integrating exact RPC camera models with 3D Gaussian Splatting.
- It employs multi-view transformer feature aggregation and a Cross-Self Consistency Module to address transient objects and radiometric inconsistencies.
- SkySplat outperforms existing methods with significantly lower MAE and faster processing, making it ideal for data-scarce remote-sensing applications.
Searching arXiv for the specified SkySplat paper and closely related splatting papers in remote sensing and feed-forward 3DGS. SkySplat is a self-supervised framework for reconstructing three-dimensional scenes from multi-temporal sparse satellite images by combining generalizable 3D Gaussian Splatting with explicit Rational Polynomial Coefficient (RPC) camera modeling. It is designed for the remote-sensing regime in which sparse geometric constraints, transient objects, and radiometric inconsistencies make conventional 3DGS pipelines and pinhole-camera assumptions inadequate. The method integrates the RPC model into the generalizable 3DGS pipeline, relies only on RGB images and radiometric-robust relative height supervision, and introduces a Cross-Self Consistency Module (CSCM) together with a multi-view consistency aggregation strategy to improve reconstruction quality and cross-dataset generalization (Huang et al., 13 Aug 2025).
1. Definition and problem setting
SkySplat addresses 3D scene reconstruction from sparse-view satellite images, with emphasis on the multi-temporal case in which images may be acquired at different dates and under different radiometric conditions. In the formulation reported for the method, existing 3D Gaussian Splatting approaches are described as unsuitable for satellite imagery because of incompatibility with RPC models and limited generalization capability, while recent generalizable 3DGS methods are reported to perform poorly on multi-temporal sparse satellite images because of limited geometric constraints, transient objects, and radiometric inconsistencies (Huang et al., 13 Aug 2025).
The framework is characterized as the first generalizable 3D Gaussian Splatting approach to explicitly model satellite-specific RPC camera models without approximation in the pipeline. It is also explicitly self-supervised: the method dispenses with ground-truth height maps and instead uses RGB imagery together with relative height supervision. This places SkySplat at the intersection of generalizable novel-view synthesis, satellite photogrammetry, and remote-sensing-specific geometric modeling (Huang et al., 13 Aug 2025).
A useful contextual distinction is that SkySplat is a satellite-only reconstruction framework, whereas later cross-view systems fuse ground and orthorectified satellite imagery in a unified coordinate frame, and other remote-sensing splatting methods focus on semantic segmentation rather than geometric reconstruction (Turkulainen et al., 19 May 2026, Qi et al., 2024). This suggests that SkySplat occupies a specific niche: generalizable, feed-forward-style satellite reconstruction under sparse and temporally heterogeneous input conditions.
2. RPC-native generalizable 3D Gaussian Splatting
The central architectural contribution is RPC-integrated generalizable 3D Gaussian Splatting. SkySplat follows a pipeline in which multi-view transformer-based feature aggregation extracts per-image features , described as following the MVSPlat approach (Huang et al., 13 Aug 2025). For each reference feature map, the method samples a set of height hypotheses and performs inverse RPC projection from image coordinates to 3D world coordinates:
$(\text{Lat}_{i}^{m}, \text{Lon}_{i}^{m}, \text{Hei}_{i}^{m}) = \text{RPC}_{\text{ref}^{-1}(u_i, v_i, h_m).$
These 3D world hypotheses are then projected into each source view using the source RPC:
$F_{j \rightarrow i}^{h_m} = \text{Inter}(F_j, \text{RPC}_{\text{src}(\text{Lat}_i^m, \text{Lon}_i^m, \text{Hei}_i^m)).$
Variance-based operations are used to construct a cost volume encoding multi-view consistency, after which a soft-argmin along the height dimension yields per-pixel height estimates (Huang et al., 13 Aug 2025).
Gaussian parameter prediction proceeds from these heights. The 3D centers are computed from the estimated heights with exact inverse RPC, while a 2D U-Net predicts additional Gaussian attributes:
Here denotes scale, rotation, 0 color coefficients, and 1 opacity. The training procedure derives 2 through a pinhole model approximation for convenience, while inference uses exact RPC (Huang et al., 13 Aug 2025).
Within the broader 3DGS landscape, this explicit camera-model integration distinguishes SkySplat from ground-image feed-forward methods and from remote-sensing splatting systems whose focus is semantic projection and segmentation rather than RPC-native geometry estimation (Zhang et al., 3 Apr 2026, Qi et al., 2024).
3. Self-supervised learning and radiometric robustness
A defining property of SkySplat is that it eliminates the need for ground-truth height maps. The reported supervision consists of a photometric term over trustworthy image regions and a radiometric-robust relative height term derived from an off-the-shelf monocular depth estimator, specifically Depth Anything V2 (DAMV2) (Huang et al., 13 Aug 2025).
The height supervision is implemented through a scale-invariant Pearson correlation loss between the predicted height 3 and a relative height map 4:
5
The use of correlation rather than absolute regression is explicitly motivated as robustness to radiometric variation and scale changes. Photometric supervision uses LPIPS and MSE, but only on pixels selected as trustworthy by the masking mechanism:
6
The overall objective is
7
This training design is presented as a response to strong radiometric inconsistencies across satellite images acquired at different times (Huang et al., 13 Aug 2025).
The elimination of LiDAR or ground-truth DSM requirements is operationally important in remote sensing. A plausible implication is that the method is intended for data-scarce settings where dense terrain supervision is unavailable, while still preserving geometric learning signals through relative height priors and masked photometric consistency (Huang et al., 13 Aug 2025).
4. Cross-Self Consistency Module and transient-object handling
Multi-temporal satellite imagery commonly contains transient objects such as moving vehicles and seasonally variable vegetation. SkySplat addresses these with the Cross-Self Consistency Module (CSCM), which computes uncertainty or confidence maps to identify transient regions and halt gradient propagation where needed (Huang et al., 13 Aug 2025).
The module uses pretrained DINOv2 and FeatUp features for robust photometric representation. For pixels deemed geometrically consistent, cross-view confidence is defined as
8
while for invalid or masked regions a self-similarity confidence is used:
9
The combined confidence map $(\text{Lat}_{i}^{m}, \text{Lon}_{i}^{m}, \text{Hei}_{i}^{m}) = \text{RPC}_{\text{ref}^{-1}(u_i, v_i, h_m).$0 is thresholded at $(\text{Lat}_{i}^{m}, \text{Lon}_{i}^{m}, \text{Hei}_{i}^{m}) = \text{RPC}_{\text{ref}^{-1}(u_i, v_i, h_m).$1 to produce a binary mask $(\text{Lat}_{i}^{m}, \text{Lon}_{i}^{m}, \text{Hei}_{i}^{m}) = \text{RPC}_{\text{ref}^{-1}(u_i, v_i, h_m).$2, and this masking is applied after 35k iterations, reported as empirically optimal (Huang et al., 13 Aug 2025).
In the method’s interpretation, transient content should not contribute equally to geometry and appearance learning. The CSCM therefore acts as a reliability filter over self-supervised correspondence signals. This mechanism differs from semantic pseudo-label propagation in remote-sensing semantic splatting, where SAM2-generated labels are introduced to improve supervision in weakly labeled boundary regions rather than to suppress transient-object corruption in geometry learning (Qi et al., 2024).
5. Multi-view consistency aggregation and reconstruction filtering
SkySplat supplements CSCM with a multi-view consistency aggregation strategy intended to minimize outliers and noise in geomodels. For each scene point $(\text{Lat}_{i}^{m}, \text{Lon}_{i}^{m}, \text{Hei}_{i}^{m}) = \text{RPC}_{\text{ref}^{-1}(u_i, v_i, h_m).$3, its 3D location is projected via RPC from a reference view to a source view and back:
$(\text{Lat}_{i}^{m}, \text{Lon}_{i}^{m}, \text{Hei}_{i}^{m}) = \text{RPC}_{\text{ref}^{-1}(u_i, v_i, h_m).$4
Two reprojection criteria are then computed:
$(\text{Lat}_{i}^{m}, \text{Lon}_{i}^{m}, \text{Hei}_{i}^{m}) = \text{RPC}_{\text{ref}^{-1}(u_i, v_i, h_m).$5
and
$(\text{Lat}_{i}^{m}, \text{Lon}_{i}^{m}, \text{Hei}_{i}^{m}) = \text{RPC}_{\text{ref}^{-1}(u_i, v_i, h_m).$6
Only points satisfying $(\text{Lat}_{i}^{m}, \text{Lon}_{i}^{m}, \text{Hei}_{i}^{m}) = \text{RPC}_{\text{ref}^{-1}(u_i, v_i, h_m).$7 pixels and $(\text{Lat}_{i}^{m}, \text{Lon}_{i}^{m}, \text{Hei}_{i}^{m}) = \text{RPC}_{\text{ref}^{-1}(u_i, v_i, h_m).$8 are retained, and these retained points are used to provide reliable splats for DSM generation (Huang et al., 13 Aug 2025).
This aggregation stage is explicitly described as a robust multi-view approach for improved geometric consensus. It functions as a post-prediction filtering mechanism over the reconstructed geometry, complementing the earlier masking of training signals performed by CSCM. The combination suggests a two-stage robustness design: first suppress unreliable supervision, then retain only geometrically stable reconstructions. That design is consistent with the paper’s emphasis on sparse geometry, temporal variability, and domain-specific camera modeling (Huang et al., 13 Aug 2025).
6. Empirical performance and comparative position
SkySplat is reported to outperform both generalizable 3DGS baselines and per-scene optimization methods. On the DFC19 benchmark, the prior state-of-the-art generalizable baseline HiSplat is reported with MAE $(\text{Lat}_{i}^{m}, \text{Lon}_{i}^{m}, \text{Hei}_{i}^{m}) = \text{RPC}_{\text{ref}^{-1}(u_i, v_i, h_m).$9, whereas SkySplat achieves MAE $F_{j \rightarrow i}^{h_m} = \text{Inter}(F_j, \text{RPC}_{\text{src}(\text{Lat}_i^m, \text{Lon}_i^m, \text{Hei}_i^m)).$0, RMSE $F_{j \rightarrow i}^{h_m} = \text{Inter}(F_j, \text{RPC}_{\text{src}(\text{Lat}_i^m, \text{Lon}_i^m, \text{Hei}_i^m)).$1, $F_{j \rightarrow i}^{h_m} = \text{Inter}(F_j, \text{RPC}_{\text{src}(\text{Lat}_i^m, \text{Lon}_i^m, \text{Hei}_i^m)).$2, and $F_{j \rightarrow i}^{h_m} = \text{Inter}(F_j, \text{RPC}_{\text{src}(\text{Lat}_i^m, \text{Lon}_i^m, \text{Hei}_i^m)).$3 (Huang et al., 13 Aug 2025). On the MVS3D test set, HiSplat is reported with MAE $F_{j \rightarrow i}^{h_m} = \text{Inter}(F_j, \text{RPC}_{\text{src}(\text{Lat}_i^m, \text{Lon}_i^m, \text{Hei}_i^m)).$4 and SkySplat with MAE $F_{j \rightarrow i}^{h_m} = \text{Inter}(F_j, \text{RPC}_{\text{src}(\text{Lat}_i^m, \text{Lon}_i^m, \text{Hei}_i^m)).$5 (Huang et al., 13 Aug 2025).
Against per-scene optimization, the comparison is made with EOGS. The reported average time for EOGS is $F_{j \rightarrow i}^{h_m} = \text{Inter}(F_j, \text{RPC}_{\text{src}(\text{Lat}_i^m, \text{Lon}_i^m, \text{Hei}_i^m)).$6 minutes per scene, whereas SkySplat requires $F_{j \rightarrow i}^{h_m} = \text{Inter}(F_j, \text{RPC}_{\text{src}(\text{Lat}_i^m, \text{Lon}_i^m, \text{Hei}_i^m)).$7 seconds per scene, yielding an $F_{j \rightarrow i}^{h_m} = \text{Inter}(F_j, \text{RPC}_{\text{src}(\text{Lat}_i^m, \text{Lon}_i^m, \text{Hei}_i^m)).$8 speedup together with higher accuracy (Huang et al., 13 Aug 2025). The reported SkySplat average MAE range is $F_{j \rightarrow i}^{h_m} = \text{Inter}(F_j, \text{RPC}_{\text{src}(\text{Lat}_i^m, \text{Lon}_i^m, \text{Hei}_i^m)).$9–0, compared with an EOGS range of approximately 1–2 (Huang et al., 13 Aug 2025).
| Benchmark or comparison | Baseline | SkySplat |
|---|---|---|
| DFC19 MAE | HiSplat: 13.18 m | 1.80 m |
| DFC19 RMSE | — | 2.68 m |
| DFC19 3 | — | 78.27% |
| DFC19 4 | — | 95.57% |
| MVS3D test MAE | HiSplat: 15.63 m | 3.42 m |
| Per-scene runtime | EOGS: 4.6 min/scene | 3.19 sec/scene |
The ablation summary states that each of the principal components—CSCM, relative height supervision, and consistency aggregation—incrementally improves performance, and that even without consistency aggregation the framework outperforms competing baselines (Huang et al., 13 Aug 2025). The paper also reports strong cross-dataset generalization on the MVS3D benchmark (Huang et al., 13 Aug 2025).
In relation to subsequent work, SkySplat differs from Cross-View Splatter, which fuses orthorectified satellite and GPS-tagged ground imagery in a unified 3D coordinate frame, and from SparseSplat, which targets adaptive compactness in feed-forward 3DGS maps using entropy-based sampling and local point-cloud networks (Turkulainen et al., 19 May 2026, Zhang et al., 3 Apr 2026). SkySplat’s comparative identity is therefore centered less on compactness or cross-modal fusion than on RPC-native satellite reconstruction under sparse, multi-temporal input.
7. Scope, limitations, and research context
The reported limitations are that SkySplat’s effectiveness still depends on camera pose and view coverage quality, and that cross-view collaboration increases computational cost (Huang et al., 13 Aug 2025). Future directions named in the source are adaptive view sampling and lightweight consistency regularization (Huang et al., 13 Aug 2025). These limitations are consistent with the method’s reliance on sparse geometric cues and with the explicit cross-view consistency machinery used during training and aggregation.
Within remote sensing more broadly, Gaussian splatting has also been adapted for efficient multi-view semantic segmentation by projecting RGB attributes and semantic features of point clouds onto the image plane and introducing SAM2-based pseudo-labeling and two-level aggregation losses (Qi et al., 2024). In parallel, cross-view georeferenced synthesis has explored joint prediction from ground and satellite imagery in a unified frame (Turkulainen et al., 19 May 2026). These adjacent developments indicate that splatting-based remote-sensing research is diversifying across reconstruction, semantics, and multimodal fusion, but SkySplat remains specifically defined by three technical commitments: exact RPC integration, self-supervision without ground-truth height maps, and robustness mechanisms for transient objects and radiometric inconsistency (Huang et al., 13 Aug 2025).
A common misconception would be to interpret SkySplat as a generic feed-forward 3DGS method for ordinary perspective imagery. The reported method is instead specialized for satellite images and their RPC geometry, and its principal innovations are tied to remote-sensing-specific failure modes rather than to generic Gaussian sparsification or cross-modal view fusion (Huang et al., 13 Aug 2025, Zhang et al., 3 Apr 2026, Turkulainen et al., 19 May 2026).