CoRe-GS: Coarse-to-Refined Gaussian Splatting

• CoRe-GS is a two-stage Gaussian Splatting method that initially uses a brief semantic phase to create a segmentation-ready, coarse 3D scene for mobile reconstruction.
• The pipeline isolates points of interest by extracting semantic labels and refining only the target object with a color-based filtering process that minimizes artifacts.
• By focusing optimization on the PoI rather than the full scene, CoRe-GS significantly reduces runtime while maintaining high PSNR/SSIM in novel view synthesis.

Searching arXiv for CoRe-GS and closely related Gaussian Splatting work to ground the article in current papers. CoRe-GS, short for Coarse-to-Refined Gaussian Splatting with Semantic Object Focus, is a two-stage 3D Gaussian Splatting pipeline for mobile 3D reconstruction with autonomous aerial robots (or similar mobile platforms) in settings such as tele-guidance and disaster response, where reconstruction quality, fast processing, and attention to specific Points of Interest (PoIs) are jointly required (Schieber et al., 5 Sep 2025). Its central premise is that, when the target object is already known or detected early, full-scene semantic optimization is often unnecessary: a short semantic stage can produce a segmentation-ready, coarse scene, after which optimization can be restricted to the target object alone. CoRe-GS therefore performs a brief semantic Gaussian Splatting stage, extracts a PoI-specific Gaussian subset, and then refines only that subset with standard Gaussian Splatting and a color-based effective filtering procedure designed to reduce floaters and boundary artifacts, yielding reduced runtime and higher or comparable PoI-focused novel-view-synthesis quality (Schieber et al., 5 Sep 2025).

1. Problem setting and design rationale

CoRe-GS is formulated for mobile reconstruction in scenarios where a robot requires usable 3D information during or shortly after data capture rather than after prolonged offline optimization (Schieber et al., 5 Sep 2025). The motivating application profile combines three constraints: accurate 3D reconstruction, high-quality novel view synthesis, and fast processing / training. A further task prior is that the system often needs to emphasize one object or semantic category rather than the entire environment.

The method is motivated by the inefficiency of uniform full-scene optimization under these conditions. Standard 3D Gaussian Splatting methods optimize all Gaussians in the scene, often for tens of thousands of iterations, even when only one object is relevant (Schieber et al., 5 Sep 2025). In such cases, much of the computation is expended on background or otherwise irrelevant regions. The paper explicitly frames full semantic training as overkill when the practical objective is a “good enough” global view together with high-quality reconstruction of one PoI (Schieber et al., 5 Sep 2025).

The method also responds to specific limitations in prior Gaussian-splatting pipelines. Standard GS (Kerbl et al.) provides strong NVS quality and speed but lacks built-in semantics, so PoI extraction depends on 2D masks and ad-hoc 3D filtering, which is noisy and offline (Schieber et al., 5 Sep 2025). Semantic GS / Gaussian Grouping (GG; Ye et al.) augments Gaussians with semantic features and supports segmentation-based editing, but its semantic rasterization and classification incur non-trivial overhead throughout training, and post-training PoI extraction can still contain floaters and boundary artifacts (Schieber et al., 5 Sep 2025). CoRe-GS addresses these issues by making semantic training early and shallow, then dropping semantics and refining only the PoI (Schieber et al., 5 Sep 2025).

2. Pipeline structure and representation

CoRe-GS is a two-stage pipeline. In the coarse semantic stage, a semantic GS model based on Gaussian Grouping (GG) is trained only briefly, for example 3000 iterations, to produce a coarse but usable scene and per-Gaussian semantic assignments sufficient for object isolation (Schieber et al., 5 Sep 2025). In the subsequent refinement stage, those semantic labels are used to identify images containing the target class, build binary masks, extract a PoI-only Gaussian subset, and initialize a new standard GS scene restricted to that object (Schieber et al., 5 Sep 2025).

The underlying scene representation follows 3D Gaussian Splatting. A scene is represented by a set of Gaussians

$\mathcal{G} = \{ G_i \}_{i=1}^N,$

where each Gaussian has a center $\mathbf{\mu}_i \in \mathbb{R}^3$, covariance represented via scaling and rotation, opacity $\alpha_i \in [0,1]$, and appearance encoded by spherical harmonics coefficients $\mathbf{c}_i$ (Schieber et al., 5 Sep 2025). Rendering is performed by projecting each 3D Gaussian into the image plane, sorting contributions front-to-back, and accumulating color through alpha compositing:

$$\mathbf{C}(x, y) = \sum_k T_k \alpha_k \mathbf{c}_k,$$

with

$T_k = \prod_{m<k}(1-\alpha_m).$

An equivalent formulation is

$$\mathbf{C} = \sum_k w_k \mathbf{c}_k, \quad w_k = \alpha_k \prod_{m<k}(1-\alpha_m)$$

(Schieber et al., 5 Sep 2025).

During the semantic stage, CoRe-GS employs Gaussian Grouping. In this formulation, each Gaussian carries semantic information, and per-pixel semantic logits are accumulated analogously to color:

$\mathbf{s}(x,y)=\sum_k w_k \mathbf{s}_k,$

where $\mathbf{s}_k$ are per-Gaussian semantic logits (Schieber et al., 5 Sep 2025). Pixel-wise semantic prediction is obtained via softmax, and optimization uses a photometric term together with semantic cross-entropy, in a total loss of the form

$\mathcal{L}_{\text{total} = \lambda_{\text{rgb} \mathcal{L}_{\text{rgb} + \lambda_{\text{sem} \mathcal{L}_{\text{sem} + \dots$

as reported in the source description (Schieber et al., 5 Sep 2025). The refinement stage discards the semantic classifier and returns to standard GS rasterization.

This architecture yields what the paper terms a coarse-to-refined with semantic object focus strategy: semantics are used only long enough to identify the object, after which all remaining optimization is concentrated on the PoI (Schieber et al., 5 Sep 2025).

3. Semantic initialization and PoI extraction

The first stage aims to produce a segmentation-ready scene rather than a fully converged semantic reconstruction (Schieber et al., 5 Sep 2025). Initialization follows the standard GS setup, using SfM/Colmap points → initial Gaussians, after which semantic GS / GG is trained on the full scene for only 3,000 iterations with photometric and semantic supervision (Schieber et al., 5 Sep 2025). The reported interpretation is that, after this short run, the scene is already visually exploitable and the semantic class IDs are stable enough for PoI extraction (Schieber et al., 5 Sep 2025).

The choice of 3000 iterations is empirical. According to the ablation summarized in the data, 1000/2000/3000/4000 initialization iterations were compared, and 3000 produced a good trade-off between training time and final PSNR/SSIM (Schieber et al., 5 Sep 2025). Visual evidence reportedly shows that <2000 iterations leads to under-segmented scenes, whereas 3000 yields clearer PoI boundaries (Schieber et al., 5 Sep 2025).

The PoI is defined as any selected semantic class ID. The examples in the reported experiments are “car” for NeRDS 360 and “chair” for SCRREAM (Schieber et al., 5 Sep 2025). PoI extraction then proceeds in three steps. First, images are filtered by class presence: if the semantic GS predicts the target class in an image, that image is retained for refinement (Schieber et al., 5 Sep 2025). Second, a binary mask

$\mathbf{\mu}_i \in \mathbb{R}^3$0

is constructed for each selected image (Schieber et al., 5 Sep 2025). These masks later restrict training losses to PoI pixels and assist in identifying PoI-associated Gaussians. Third, a per-Gaussian filtering step keeps Gaussians whose semantic label corresponds to the PoI class, yielding a PoI-only Gaussian set (Schieber et al., 5 Sep 2025).

A plausible implication is that CoRe-GS treats semantic prediction not as an end in itself but as an intermediate localization mechanism. The semantic stage therefore need not achieve full-scene convergence, only enough semantic stability to support reliable object extraction.

4. Color-based effective filtering and refinement dynamics

The principal methodological novelty beyond simple semantic filtering is the color-based effective filtering used during refinement (Schieber et al., 5 Sep 2025). The paper motivates this module by observing that post-hoc PoI extraction from semantic GS can leave floaters and artifacts near object boundaries, while continuing semantic rasterization during refinement would preserve unwanted computational overhead (Schieber et al., 5 Sep 2025).

The method begins with pre-extraction color analysis on each selected PoI-containing image. The image is first downscaled by 0.5, then the set of unique colors

$\mathbf{\mu}_i \in \mathbb{R}^3$1

is extracted, and a KD-tree is built over $\mathbf{\mu}_i \in \mathbb{R}^3$2 (Schieber et al., 5 Sep 2025). A predefined reduced RGB color space

$\mathbf{\mu}_i \in \mathbb{R}^3$3

is then sampled, and for each candidate color $\mathbf{\mu}_i \in \mathbb{R}^3$4 the minimum distance to any color in the image is computed:

$\mathbf{\mu}_i \in \mathbb{R}^3$5

The method identifies the color

$\mathbf{\mu}_i \in \mathbb{R}^3$6

that is furthest from all image colors, where

$\mathbf{\mu}_i \in \mathbb{R}^3$7

with $\mathbf{\mu}_i \in \mathbb{R}^3$8 (Schieber et al., 5 Sep 2025). This $\mathbf{\mu}_i \in \mathbb{R}^3$9 is the image’s “furthest color”. The method also tracks an average distance $\alpha_i \in [0,1]$0, which is later used to define filtering thresholds (Schieber et al., 5 Sep 2025).

During refinement, scene filtering is applied every 1000 iterations (Schieber et al., 5 Sep 2025). For each view, each Gaussian’s rendered color $\alpha_i \in [0,1]$1 is obtained from its spherical harmonics parameters and the camera view direction. This color is compared to a corresponding background color $\alpha_i \in [0,1]$2 linked to the furthest-color analysis, using the distance

$\alpha_i \in [0,1]$3

A removal threshold is defined as

$\alpha_i \in [0,1]$4

with default $\alpha_i \in [0,1]$5 (Schieber et al., 5 Sep 2025). Gaussians are then removed if

$\alpha_i \in [0,1]$6

The stated interpretation is that Gaussians too close in color to the determined “background / outlier” reference are treated as artifacts and pruned (Schieber et al., 5 Sep 2025).

The refinement stage itself initializes a new GS scene using only the filtered PoI Gaussians, restricts the training images to those containing the PoI, and computes photometric loss only on masked pixels:

$\alpha_i \in [0,1]$7

where $\alpha_i \in [0,1]$8 is described as, for example, L2 + fused-SSIM (Schieber et al., 5 Sep 2025). Optimization uses a Sparse Adam optimizer and Fusion SSIM loss, and the reported default schedule is 27k iterations after the initial 3k semantic iterations (Schieber et al., 5 Sep 2025). The effect is a PoI-restricted optimization in which background Gaussians are no longer trained, and periodic color-based pruning reduces spurious structure around the object (Schieber et al., 5 Sep 2025).

5. Training schedule, implementation, and runtime profile

The overall schedule consists of three phases: a coarse semantic stage, a PoI extraction step, and a PoI refinement stage (Schieber et al., 5 Sep 2025). The semantic stage lasts about 3,000 iterations; PoI extraction takes around 4 seconds per scene; and refinement then runs for about 27,000 iterations, with color-based filtering every 1000 iterations (Schieber et al., 5 Sep 2025).

The implementation is reported in Python + PyTorch + CUDA, and experiments were performed on an NVIDIA RTX 3090 (24 GB VRAM) (Schieber et al., 5 Sep 2025). Furthest-color computation over views is parallelized with a thread pool, while GS rasterization and fused losses run on the GPU (Schieber et al., 5 Sep 2025).

The runtime comparison reported in the paper is central to the method’s motivation.

Dataset / method	Time per scene	Notes
NeRDS 360, GG full training	2050 s	~30k iterations
NeRDS 360, CoRe-GS	460 s	~120 s semantic init, ~4 s extraction, remainder refinement
SCRREAM, GG	2675 s	semantic GS baseline
SCRREAM, CoRe-GS	816 s	hybrid two-stage pipeline

On NeRDS 360, this corresponds to approximately 4.5× faster runtime, described as about a quarter of GG runtime (Schieber et al., 5 Sep 2025). On SCRREAM, the reduction is about 3.3× (Schieber et al., 5 Sep 2025). The data block further states that CoRe-GS therefore effectively cuts training time to roughly a quarter of semantic GS (Schieber et al., 5 Sep 2025).

A notable aspect of this schedule is that refinement uses standard GS rasterization rather than semantic rasterization. This suggests that much of the runtime reduction derives not only from optimizing fewer Gaussians, but also from eliminating semantic computation after the object has been localized.

6. Experimental evaluation

CoRe-GS is evaluated on two datasets: NeRDS 360 (Neo 360) – outdoor, synthetic street scenes and SCRREAM – indoor, real-world (Schieber et al., 5 Sep 2025). For NeRDS 360, the used scenes are listed as SF_6thAndMission_medium: views 6, 7, 10; SF_GrantAndCalifornia: views 1, 2, 3; SF_VanNessAveAndTurkSt: views 3, 5, and the PoI class is “car” (Schieber et al., 5 Sep 2025). For SCRREAM, the scenes are scene01_full_00, scene02_full_00, scene03_full_00, with PoI class “chair” (Schieber et al., 5 Sep 2025).

Evaluation uses PSNR, SSIM, and runtime, with PoI-focused metrics often masked by PoI ground truth to ensure fair comparison when methods differ in their ability to remove background (Schieber et al., 5 Sep 2025).

Comparison with Gaussian Grouping

On NeRDS 360, three variants are compared: GG (convex hull post-processing), GG (direct object removal), and CoRe-GS (Schieber et al., 5 Sep 2025). The reported mean results are:

Method	PSNR	SSIM	Time
GG (convex-hull)	11.238	0.748	2050 s
GG (direct removal)	19.389	0.870	2050 s
CoRe-GS	27.204	0.963	460 s

These values indicate that CoRe-GS is both faster and substantially stronger on PoI-focused NVS for the outdoor synthetic scenes (Schieber et al., 5 Sep 2025). The source description attributes especially poor performance of convex-hull removal to over-removal (Schieber et al., 5 Sep 2025).

On SCRREAM, the comparison is between GG (direct removal) and CoRe-GS (Schieber et al., 5 Sep 2025). Reported per-scene and mean values are:

Scene / mean	GG PSNR / SSIM / time	CoRe-GS PSNR / SSIM / time
scene01	31.90 / 0.927 / 2614 s	30.89 / 0.903 / 709 s
scene02	25.61 / 0.849 / 2766 s	25.97 / 0.792 / 1083 s
scene03	36.52 / 0.965 / 2645 s	35.79 / 0.954 / 656 s
mean	31.34 / 0.914 / 2675 s	30.88 / 0.883 / 816 s

The reported interpretation is that CoRe-GS is essentially equal in mean PSNR while being much faster, though mean SSIM is lower than GG on SCRREAM (Schieber et al., 5 Sep 2025). Qualitative figures are said to show fewer floaters and sharper boundaries for CoRe-GS on both NeRDS 360 and SCRREAM (Schieber et al., 5 Sep 2025).

Initialization ablation

An ablation on SF_6thAndMission_medium10 examines initialization lengths of 1000/2000/3000/4000 semantic iterations at 30k total iterations (Schieber et al., 5 Sep 2025). The reported values are:

Semantic init iterations	PSNR	SSIM	Time
1000	27.107	0.962	380 s
2000	27.542	0.965	391 s
3000	27.668	0.965	447 s
4000	27.553	0.964	480 s

These numbers support the choice of 3000 iterations as a quality/time equilibrium (Schieber et al., 5 Sep 2025).

Comparison with standard GS

The data also compare standard GS (30k iterations, full scene) with CoRe-GS (15k iterations total, but only PoI training; metrics masked) (Schieber et al., 5 Sep 2025). The reported values are:

Method	PSNR	SSIM	Time
Standard GS (30k)	21.40	0.862	417 s
CoRe-GS (15k)	21.73	0.859	285 s

These results are presented as evidence that, even with half the iterations, CoRe-GS attains slightly better PoI PSNR and similar SSIM with lower runtime (Schieber et al., 5 Sep 2025).

7. Relation to adjacent methods, scope, and limitations

CoRe-GS is situated relative to several lines of work. With respect to standard GS (Kerbl et al.), it preserves the Gaussian-splatting representation and real-time rendering orientation but introduces an object-centric schedule that uses semantics only during an initial stage (Schieber et al., 5 Sep 2025). Relative to Gaussian Grouping and GAGA (Lyu et al.), it shares the semantic foundation but differs in three stated ways: a coarse-to-fine schedule, object-focused refinement in which only PoI Gaussians are optimized, and color-based effective filtering to remove floaters without semantic computation during refinement (Schieber et al., 5 Sep 2025). The stated target is efficiency and PoI quality, rather than full-scene semantic editing (Schieber et al., 5 Sep 2025).

The method is also distinguished from object-removal and active-view-selection approaches such as OR-NeRF, Removing Objects from NeRF, FisherRF, and ActiveSplat, as well as language-guided or semantic radiance-field methods such as LERF, LEGS, HAMMER, Object-Aware GS, and Graspsplats (Schieber et al., 5 Sep 2025). The paper’s framing is that CoRe-GS is not about acquiring more views, but about using existing views more efficiently when the PoI is already known (Schieber et al., 5 Sep 2025). It is also described as modular, in the sense that any semantic GS backend could be used for the coarse stage (Schieber et al., 5 Sep 2025).

Several limitations are explicitly identified. First, CoRe-GS depends on segmentation quality: poor semantic labels lead to poor PoI extraction (Schieber et al., 5 Sep 2025). Second, the reported evaluation is mainly single-PoI per run; multiple PoIs would require repeated runs or further adaptation (Schieber et al., 5 Sep 2025). Third, the color-based filtering assumptions require that color differences between PoI, background, and artifacts be sufficiently distinguishable in RGB space (Schieber et al., 5 Sep 2025).

The title similarity between CoRe-GS (Schieber et al., 5 Sep 2025), CoR-GS (Zhang et al., 2024), and CORE (Bo et al., 15 Dec 2025) can invite confusion, but they address different problems. CoR-GS is a sparse-view 3D Gaussian Splatting method based on co-regularization between two independently trained radiance fields, using co-pruning and pseudo-view co-regularization to combat sparse-view overfitting (Zhang et al., 2024). CORE is a graph self-supervised learning framework, Contrastive Masked Feature Reconstruction, formulated for graphs rather than 3D scene reconstruction (Bo et al., 15 Dec 2025). CoRe-GS, by contrast, is explicitly a semantic-object-focused Gaussian-splatting pipeline for mobile reconstruction (Schieber et al., 5 Sep 2025).

Taken together, the reported results suggest that CoRe-GS is best understood as an object-centric training schedule for semantic Gaussian Splatting: semantics are used briefly to localize the target, then discarded in favor of faster standard-GS optimization on a PoI-only subset. A plausible implication is that its main contribution lies less in changing the Gaussian representation than in reorganizing when and where computational effort is spent.