---
title: 'CoRe-GS: Coarse-to-Refined Gaussian Splatting'
url: https://www.emergentmind.com/topics/core-gs
type: topic
---

# CoRe-GS: Coarse-to-Refined Gaussian Splatting

Searching arXiv for CoRe-GS and closely related Gaussian Splatting work to ground the article in current papers.
CoRe-GS, short for **Coarse-to-Refined Gaussian Splatting with Semantic Object Focus**, is a two-stage 3D Gaussian Splatting pipeline for **mobile 3D reconstruction with autonomous aerial robots (or similar mobile platforms)** in settings such as tele-guidance and disaster response, where reconstruction quality, fast processing, and attention to specific **Points of Interest (PoIs)** are jointly required [2509.04859]. Its central premise is that, when the target object is already known or detected early, full-scene semantic optimization is often unnecessary: a short semantic stage can produce a **segmentation-ready, coarse scene**, after which optimization can be restricted to the target object alone. CoRe-GS therefore performs a brief semantic Gaussian Splatting stage, extracts a PoI-specific Gaussian subset, and then refines only that subset with standard Gaussian Splatting and a **color-based effective filtering** procedure designed to reduce floaters and boundary artifacts, yielding reduced runtime and higher or comparable PoI-focused novel-view-synthesis quality [2509.04859].

## 1. Problem setting and design rationale

CoRe-GS is formulated for **mobile reconstruction** in scenarios where a robot requires usable 3D information during or shortly after data capture rather than after prolonged offline optimization [2509.04859]. The motivating application profile combines three constraints: **accurate 3D reconstruction**, high-quality novel view synthesis, and **fast processing / training**. A further task prior is that the system often needs to emphasize one object or semantic category rather than the entire environment.

The method is motivated by the inefficiency of uniform full-scene optimization under these conditions. Standard 3D Gaussian Splatting methods optimize **all Gaussians in the scene**, often for **tens of thousands of iterations**, even when only one object is relevant [2509.04859]. In such cases, much of the computation is expended on background or otherwise irrelevant regions. The paper explicitly frames full semantic training as **overkill** when the practical objective is a “good enough” global view together with **high-quality reconstruction of one PoI** [2509.04859].

The method also responds to specific limitations in prior Gaussian-splatting pipelines. **Standard GS (Kerbl et al.)** provides strong NVS quality and speed but lacks built-in semantics, so PoI extraction depends on 2D masks and ad-hoc 3D filtering, which is noisy and offline [2509.04859]. **Semantic GS / Gaussian Grouping (GG; Ye et al.)** augments Gaussians with semantic features and supports segmentation-based editing, but its semantic rasterization and classification incur non-trivial overhead throughout training, and post-training PoI extraction can still contain floaters and boundary artifacts [2509.04859]. CoRe-GS addresses these issues by making semantic training **early and shallow**, then **dropping semantics** and refining only the PoI [2509.04859].

## 2. Pipeline structure and representation

CoRe-GS is a **two-stage** pipeline. In the **coarse semantic stage**, a semantic GS model based on **Gaussian Grouping (GG)** is trained only briefly, for example **3000 iterations**, to produce a coarse but usable scene and per-Gaussian semantic assignments sufficient for object isolation [2509.04859]. In the subsequent **refinement stage**, those semantic labels are used to identify images containing the target class, build binary masks, extract a PoI-only Gaussian subset, and initialize a new standard GS scene restricted to that object [2509.04859].

The underlying scene representation follows 3D Gaussian Splatting. A scene is represented by a set of Gaussians
$$
\mathcal{G} = \{ G_i \}_{i=1}^N,
$$
where each Gaussian has a center $\mathbf{\mu}_i \in \mathbb{R}^3$, covariance represented via scaling and rotation, opacity $\alpha_i \in [0,1]$, and appearance encoded by spherical harmonics coefficients $\mathbf{c}_i$ [2509.04859]. Rendering is performed by projecting each 3D Gaussian into the image plane, sorting contributions front-to-back, and accumulating color through alpha compositing:
$$
\mathbf{C}(x, y) = \sum_k T_k \alpha_k \mathbf{c}_k,
$$
with
$$
T_k = \prod_{m<k}(1-\alpha_m).
$$
An equivalent formulation is
$$
\mathbf{C} = \sum_k w_k \mathbf{c}_k, \quad
w_k = \alpha_k \prod_{m<k}(1-\alpha_m)
$$
[2509.04859].

During the semantic stage, CoRe-GS employs **Gaussian Grouping**. In this formulation, each Gaussian carries semantic information, and per-pixel semantic logits are accumulated analogously to color:
$$
\mathbf{s}(x,y)=\sum_k w_k \mathbf{s}_k,
$$
where $\mathbf{s}_k$ are per-Gaussian semantic logits [2509.04859]. Pixel-wise semantic prediction is obtained via softmax, and optimization uses a photometric term together with semantic cross-entropy, in a total loss of the form
$$
\mathcal{L}_{\text{total} = \lambda_{\text{rgb} \mathcal{L}_{\text{rgb} + \lambda_{\text{sem} \mathcal{L}_{\text{sem} + \dots
$$
as reported in the source description [2509.04859]. The refinement stage discards the semantic classifier and returns to **standard GS rasterization**.

This architecture yields what the paper terms a **coarse-to-refined with semantic object focus** strategy: semantics are used only long enough to identify the object, after which all remaining optimization is concentrated on the PoI [2509.04859].

## 3. Semantic initialization and PoI extraction

The first stage aims to produce a **segmentation-ready** scene rather than a fully converged semantic reconstruction [2509.04859]. Initialization follows the standard GS setup, using **SfM/Colmap points → initial Gaussians**, after which semantic GS / GG is trained on the full scene for only **3,000 iterations** with photometric and semantic supervision [2509.04859]. The reported interpretation is that, after this short run, the scene is already **visually exploitable** and the semantic class IDs are **stable enough** for PoI extraction [2509.04859].

The choice of 3000 iterations is empirical. According to the ablation summarized in the data, **1000/2000/3000/4000** initialization iterations were compared, and **3000** produced a good trade-off between training time and final PSNR/SSIM [2509.04859]. Visual evidence reportedly shows that **<2000 iterations** leads to under-segmented scenes, whereas 3000 yields clearer PoI boundaries [2509.04859].

The PoI is defined as any selected semantic class ID. The examples in the reported experiments are **“car”** for NeRDS 360 and **“chair”** for SCRREAM [2509.04859]. PoI extraction then proceeds in three steps. First, images are filtered by **class presence**: if the semantic GS predicts the target class in an image, that image is retained for refinement [2509.04859]. Second, a binary mask
$$
M(u,v)=
\begin{cases}
1, & \text{if pixel } (u,v) \text{ belongs to the target object ID} \\
0, & \text{otherwise}
\end{cases}
$$
is constructed for each selected image [2509.04859]. These masks later restrict training losses to PoI pixels and assist in identifying PoI-associated Gaussians. Third, a per-Gaussian filtering step keeps Gaussians whose semantic label corresponds to the PoI class, yielding a **PoI-only Gaussian set** [2509.04859].

A plausible implication is that CoRe-GS treats semantic prediction not as an end in itself but as an intermediate localization mechanism. The semantic stage therefore need not achieve full-scene convergence, only enough semantic stability to support reliable object extraction.

## 4. Color-based effective filtering and refinement dynamics

The principal methodological novelty beyond simple semantic filtering is the **color-based effective filtering** used during refinement [2509.04859]. The paper motivates this module by observing that post-hoc PoI extraction from semantic GS can leave **floaters** and artifacts near object boundaries, while continuing semantic rasterization during refinement would preserve unwanted computational overhead [2509.04859].

The method begins with **pre-extraction color analysis** on each selected PoI-containing image. The image is first **downscaled by 0.5**, then the set of unique colors
$$
C = \{ c_i \}_{i=1}^{N}, \quad c_i \in [0,1]^3
$$
is extracted, and a **KD-tree** is built over $C$ [2509.04859]. A **predefined reduced RGB color space**
$$
P = \{ p_j \}_{j=1}^{J}, \quad p_j \in [0,1]^3
$$
is then sampled, and for each candidate color $p_j$ the minimum distance to any color in the image is computed:
$$
d_{\min}(p_j) = \min_{c_i \in C} \| p_j - c_i \|_2.
$$
The method identifies the color
$$
p^* = \arg\max_{p_j \in P} d_{\min}(p_j),
$$
that is furthest from all image colors, where
$$
\| p_j - c_i \|_2 = \sqrt{(r_{p_j} - r_{c_i})^2 + (g_{p_j} - g_{c_i})^2 + (b_{p_j} - b_{c_i})^2}
$$
with $r,g,b \in [0,1]$ [2509.04859]. This $p^*$ is the image’s **“furthest color”**. The method also tracks an average distance $d_{\text{avg}}$, which is later used to define filtering thresholds [2509.04859].

During refinement, scene filtering is applied every **1000 iterations** [2509.04859]. For each view, each Gaussian’s rendered color $c_{G_i}$ is obtained from its spherical harmonics parameters and the camera view direction. This color is compared to a corresponding background color $c_{\text{bg}}$ linked to the furthest-color analysis, using the distance
$$
d(c_{G_i}, c_{\text{bg}) = \| c_{G_i} - c_{\text{bg} \|_2.
$$
A removal threshold is defined as
$$
d_{\text{remove} = t_r \cdot d_{\text{avg}
$$
with default $t_r = 0.5$ [2509.04859]. Gaussians are then removed if
$$
G_{\text{remove} = \{ G_i \mid d(c_{G_i}, c_{\text{bg}) < d_{\text{remove} \}.
$$
The stated interpretation is that Gaussians too close in color to the determined “background / outlier” reference are treated as artifacts and pruned [2509.04859].

The refinement stage itself initializes a new GS scene using only the filtered PoI Gaussians, restricts the training images to those containing the PoI, and computes photometric loss only on masked pixels:
$$
\mathcal{L}_{\text{rgb} = \sum_{u,v} M(u,v) \cdot \ell\big( I_{\text{render}(u,v), I_{\text{gt}(u,v) \big)
$$
where $\ell$ is described as, for example, **L2 + fused-SSIM** [2509.04859]. Optimization uses a **Sparse Adam optimizer** and **Fusion SSIM loss**, and the reported default schedule is **27k iterations** after the initial 3k semantic iterations [2509.04859]. The effect is a PoI-restricted optimization in which background Gaussians are no longer trained, and periodic color-based pruning reduces spurious structure around the object [2509.04859].

## 5. Training schedule, implementation, and runtime profile

The overall schedule consists of three phases: a **coarse semantic stage**, a **PoI extraction** step, and a **PoI refinement stage** [2509.04859]. The semantic stage lasts about **3,000 iterations**; PoI extraction takes **around 4 seconds** per scene; and refinement then runs for about **27,000 iterations**, with color-based filtering every **1000 iterations** [2509.04859].

The implementation is reported in **Python + PyTorch + CUDA**, and experiments were performed on an **NVIDIA RTX 3090 (24 GB VRAM)** [2509.04859]. Furthest-color computation over views is parallelized with a thread pool, while GS rasterization and fused losses run on the GPU [2509.04859].

The runtime comparison reported in the paper is central to the method’s motivation.

| Dataset / method | Time per scene | Notes |
|---|---:|---|
| NeRDS 360, GG full training | **2050 s** | ~30k iterations |
| NeRDS 360, CoRe-GS | **460 s** | ~120 s semantic init, ~4 s extraction, remainder refinement |
| SCRREAM, GG | **2675 s** | semantic GS baseline |
| SCRREAM, CoRe-GS | **816 s** | hybrid two-stage pipeline |

On **NeRDS 360**, this corresponds to approximately **4.5× faster** runtime, described as **about a quarter** of GG runtime [2509.04859]. On **SCRREAM**, the reduction is about **3.3×** [2509.04859]. The data block further states that CoRe-GS therefore effectively **cuts training time to roughly a quarter** of semantic GS [2509.04859].

A notable aspect of this schedule is that refinement uses **standard GS rasterization** rather than semantic rasterization. This suggests that much of the runtime reduction derives not only from optimizing fewer Gaussians, but also from eliminating semantic computation after the object has been localized.

## 6. Experimental evaluation

CoRe-GS is evaluated on two datasets: **NeRDS 360 (Neo 360) – outdoor, synthetic street scenes** and **SCRREAM – indoor, real-world** [2509.04859]. For NeRDS 360, the used scenes are listed as **SF\_6thAndMission\_medium: views 6, 7, 10; SF\_GrantAndCalifornia: views 1, 2, 3; SF\_VanNessAveAndTurkSt: views 3, 5**, and the PoI class is **“car”** [2509.04859]. For SCRREAM, the scenes are **scene01_full_00, scene02_full_00, scene03_full_00**, with PoI class **“chair”** [2509.04859].

Evaluation uses **PSNR**, **SSIM**, and **runtime**, with PoI-focused metrics often masked by PoI ground truth to ensure fair comparison when methods differ in their ability to remove background [2509.04859].

### Comparison with Gaussian Grouping

On **NeRDS 360**, three variants are compared: **GG (convex hull post-processing)**, **GG (direct object removal)**, and **CoRe-GS** [2509.04859]. The reported mean results are:

| Method | PSNR | SSIM | Time |
|---|---:|---:|---:|
| GG (convex-hull) | **11.238** | 0.748 | **2050 s** |
| GG (direct removal) | **19.389** | 0.870 | **2050 s** |
| CoRe-GS | **27.204** | **0.963** | **460 s** |

These values indicate that CoRe-GS is both faster and substantially stronger on PoI-focused NVS for the outdoor synthetic scenes [2509.04859]. The source description attributes especially poor performance of convex-hull removal to over-removal [2509.04859].

On **SCRREAM**, the comparison is between **GG (direct removal)** and **CoRe-GS** [2509.04859]. Reported per-scene and mean values are:

| Scene / mean | GG PSNR / SSIM / time | CoRe-GS PSNR / SSIM / time |
|---|---|---|
| scene01 | 31.90 / 0.927 / 2614 s | 30.89 / 0.903 / **709 s** |
| scene02 | 25.61 / 0.849 / 2766 s | **25.97** / 0.792 / **1083 s** |
| scene03 | 36.52 / 0.965 / 2645 s | 35.79 / 0.954 / **656 s** |
| mean | 31.34 / 0.914 / **2675 s** | **30.88** / 0.883 / **816 s** |

The reported interpretation is that CoRe-GS is **essentially equal** in mean PSNR while being much faster, though mean SSIM is lower than GG on SCRREAM [2509.04859]. Qualitative figures are said to show fewer floaters and sharper boundaries for CoRe-GS on both NeRDS 360 and SCRREAM [2509.04859].

### Initialization ablation

An ablation on **SF\_6thAndMission\_medium10** examines initialization lengths of **1000/2000/3000/4000** semantic iterations at **30k total iterations** [2509.04859]. The reported values are:

| Semantic init iterations | PSNR | SSIM | Time |
|---|---:|---:|---:|
| 1000 | 27.107 | 0.962 | 380 s |
| 2000 | 27.542 | 0.965 | 391 s |
| 3000 | **27.668** | 0.965 | 447 s |
| 4000 | 27.553 | 0.964 | 480 s |

These numbers support the choice of **3000** iterations as a quality/time equilibrium [2509.04859].

### Comparison with standard GS

The data also compare **standard GS (30k iterations, full scene)** with **CoRe-GS (15k iterations total, but only PoI training; metrics masked)** [2509.04859]. The reported values are:

| Method | PSNR | SSIM | Time |
|---|---:|---:|---:|
| Standard GS (30k) | 21.40 | 0.862 | 417 s |
| CoRe-GS (15k) | **21.73** | 0.859 | **285 s** |

These results are presented as evidence that, even with **half the iterations**, CoRe-GS attains slightly better PoI PSNR and similar SSIM with lower runtime [2509.04859].

## 7. Relation to adjacent methods, scope, and limitations

CoRe-GS is situated relative to several lines of work. With respect to **standard GS (Kerbl et al.)**, it preserves the Gaussian-splatting representation and real-time rendering orientation but introduces an object-centric schedule that uses semantics only during an initial stage [2509.04859]. Relative to **Gaussian Grouping** and **GAGA (Lyu et al.)**, it shares the semantic foundation but differs in three stated ways: a **coarse-to-fine schedule**, **object-focused refinement** in which only PoI Gaussians are optimized, and **color-based effective filtering** to remove floaters without semantic computation during refinement [2509.04859]. The stated target is **efficiency and PoI quality**, rather than full-scene semantic editing [2509.04859].

The method is also distinguished from object-removal and active-view-selection approaches such as **OR-NeRF**, **Removing Objects from NeRF**, **FisherRF**, and **ActiveSplat**, as well as language-guided or semantic radiance-field methods such as **LERF**, **LEGS**, **HAMMER**, **Object-Aware GS**, and **Graspsplats** [2509.04859]. The paper’s framing is that CoRe-GS is **not about acquiring more views**, but about using existing views more efficiently when the PoI is already known [2509.04859]. It is also described as **modular**, in the sense that any semantic GS backend could be used for the coarse stage [2509.04859].

Several limitations are explicitly identified. First, CoRe-GS depends on **segmentation quality**: poor semantic labels lead to poor PoI extraction [2509.04859]. Second, the reported evaluation is mainly **single-PoI per run**; multiple PoIs would require repeated runs or further adaptation [2509.04859]. Third, the **color-based filtering assumptions** require that color differences between PoI, background, and artifacts be sufficiently distinguishable in RGB space [2509.04859].

The title similarity between **CoRe-GS** [2509.04859], **CoR-GS** [2405.12110], and **CORE** [2512.13235] can invite confusion, but they address different problems. **CoR-GS** is a sparse-view 3D Gaussian Splatting method based on co-regularization between two independently trained radiance fields, using **co-pruning** and **pseudo-view co-regularization** to combat sparse-view overfitting [2405.12110]. **CORE** is a graph self-supervised learning framework, **Contrastive Masked Feature Reconstruction**, formulated for graphs rather than 3D scene reconstruction [2512.13235]. CoRe-GS, by contrast, is explicitly a semantic-object-focused Gaussian-splatting pipeline for mobile reconstruction [2509.04859].

Taken together, the reported results suggest that CoRe-GS is best understood as an **object-centric training schedule for semantic Gaussian Splatting**: semantics are used briefly to localize the target, then discarded in favor of faster standard-GS optimization on a PoI-only subset. A plausible implication is that its main contribution lies less in changing the Gaussian representation than in reorganizing when and where computational effort is spent.

Source: https://www.emergentmind.com/topics/core-gs