Colored Gaussian Primitives Overview

• Colored Gaussian Primitives are explicit Gaussian-based representations defined by spatial parameters, opacity, and color attributes that enable detailed scene rendering and editing.
• They support diverse formulations including anisotropic 3D splatting, spatial color variations, and layered intrinsic decompositions to capture view-dependent effects.
• Applications span novel-view synthesis, relighting, and surgical reconstruction while addressing efficiency and edit specificity through advanced optimization techniques.

Searching arXiv for papers on colored Gaussian primitives and closely related Gaussian-splatting representations. Colored Gaussian Primitives are explicit Gaussian-based scene or color representations in which each primitive carries appearance parameters in addition to geometry and opacity. In the contemporary Gaussian-splatting literature, the term most commonly denotes anisotropic 3D Gaussian elements endowed with RGB color, opacity, and sometimes view-dependent appearance, spatially varying intra-primitive color, or physically motivated reflection and transmittance attributes. Across recent work, these primitives appear in several distinct but related roles: as the standard appearance-carrying atoms of 3D Gaussian Splatting (3DGS), as independently parameterized layers for intrinsic decomposition and editing, as ray-traceable ellipsoids or flat mesh-backed patches for light transport, as primitive-space entities for change detection, and as localized Gaussian functions in RGB space for continuous color transformation (Lanvin et al., 30 Jun 2026, Byrski et al., 31 Jan 2025, Galappaththige et al., 8 May 2026, Zeng et al., 1 Apr 2026, Xue et al., 19 May 2026).

1. Definition and core parameterizations

In standard 3DGS-style formulations, a colored Gaussian primitive is a 3D Gaussian specified by spatial parameters and appearance parameters. A representative parameterization is

$g = \{c, \alpha, p, \Sigma\},$

where $p \in \mathbb{R}^3$ is the center, $\Sigma \in \mathbb{R}^{3\times 3}$ the covariance, $\alpha \in [0,1]$ the opacity, and $c$ the color (Lanvin et al., 30 Jun 2026). In baseline 3DGS, the color is typically view-dependent and encoded with spherical harmonics (SH), whereas several later variants replace or augment SH with RGB-only color, spatially varying functions, or physically structured appearance models (Lanvin et al., 30 Jun 2026, Byrski et al., 31 Jan 2025, Xu et al., 2024).

A common covariance factorization writes

$\Sigma = R S S^{\top} R^{\top},$

with $R$ a rotation and $S$ a diagonal scaling matrix (Byrski et al., 31 Jan 2025, Ji et al., 26 Aug 2025, Galappaththige et al., 8 May 2026). This decomposition supports anisotropy and is used both in rasterization-based splatting and in ray-tracing formulations. In SafeguardGS, the same 3DGS primitive is described through position $x_i \in \mathbb{R}^3$, anisotropic scale $s_i \in \mathbb{R}^3$, quaternion rotation $p \in \mathbb{R}^3$0, opacity logit $p \in \mathbb{R}^3$1, and spherical harmonics appearance coefficients $p \in \mathbb{R}^3$2, with SH degree 3 and a total of 59 parameters per primitive (Lee et al., 2024).

Several works explicitly modify this baseline definition. RaySplats replaces SH appearance with per-primitive RGB color and ray traces 3D confidence ellipsoids directly in 3D: $p \in \mathbb{R}^3$3 where $p \in \mathbb{R}^3$4 is position, $p \in \mathbb{R}^3$5 covariance, $p \in \mathbb{R}^3$6 trainable opacity, and $p \in \mathbb{R}^3$7 RGB color (Byrski et al., 31 Jan 2025). REdiSplats instead uses flat Gaussians, forcing one scale to $p \in \mathbb{R}^3$8, and realizes each as a triangulated octagon for ray tracing and mesh-driven editing (Byrski et al., 15 Mar 2025). ColorGS defines “Colored Gaussian Primitives” by augmenting each 3D Gaussian with $p \in \mathbb{R}^3$9 screen-space anchors $\Sigma \in \mathbb{R}^{3\times 3}$0, each with learnable color parameter $\Sigma \in \mathbb{R}^{3\times 3}$1, so that intra-primitive color varies with the rendering-plane intersection point $\Sigma \in \mathbb{R}^{3\times 3}$2 (Ji et al., 26 Aug 2025).

The phrase “Colored Gaussian Primitives” also appears outside scene radiance fields. GLUT models color transformations in the RGB cube $\Sigma \in \mathbb{R}^{3\times 3}$3 using Gaussian primitives whose mean $\Sigma \in \mathbb{R}^{3\times 3}$4, covariance $\Sigma \in \mathbb{R}^{3\times 3}$5, opacity $\Sigma \in \mathbb{R}^{3\times 3}$6, and local affine transform $\Sigma \in \mathbb{R}^{3\times 3}$7 define a localized color-space mapping rather than a 3D scene element (Xue et al., 19 May 2026). This suggests that the defining feature is not a single rendering formalism, but the use of explicit Gaussian-localized appearance carriers with interpretable parameters.

2. Rendering and compositing in Gaussian-splatting pipelines

In rasterization-based 3DGS, each 3D Gaussian is projected to the image plane as a 2D Gaussian footprint, and contributions are accumulated with front-to-back alpha compositing. For a pixel $\Sigma \in \mathbb{R}^{3\times 3}$8, sorted splats $\Sigma \in \mathbb{R}^{3\times 3}$9, transmittance is

$\alpha \in [0,1]$0

and composite color is

$\alpha \in [0,1]$1

where $\alpha \in [0,1]$2 is projected opacity contribution and $\alpha \in [0,1]$3 is the view-dependent color for camera direction $\alpha \in [0,1]$4 (Lanvin et al., 30 Jun 2026). The same compositing structure appears in SafeguardGS, where $\alpha \in [0,1]$5, $\alpha \in [0,1]$6, and

$\alpha \in [0,1]$7

with $\alpha \in [0,1]$8 computed from SH for the current view (Lee et al., 2024).

ColorGS preserves this front-to-back rasterization pipeline but replaces the fixed per-Gaussian color with a spatially adaptive function

$\alpha \in [0,1]$9

where $c$0 is the rendering-plane intersection and $c$1 the ray direction. Rendering becomes

$c$2

while depth rendering remains unchanged (Ji et al., 26 Aug 2025). SuperGaussians follows an analogous strategy, defining

$c$3

thereby turning color and opacity into spatial fields over the primitive footprint (Xu et al., 2024).

Ray-tracing formulations alter the intersection model while largely retaining ordered alpha compositing. RaySplats computes intersection against 3D confidence ellipsoids

$c$4

rather than projected 2D ellipses, and accumulates color along the ray as

$c$5

(Byrski et al., 31 Jan 2025). Stochastic Ray Tracing of Transparent 3D Gaussians retains the same target compositing weights,

$c$6

but replaces explicit sorting and full accumulation by a stochastic binary-opacity acceptance process whose shading probability is exactly $c$7 (Sun et al., 9 Apr 2025).

This shared compositing structure is central to later editing and pruning work. It also underlies a recurrent misconception: sparse primitive-level color or palette decompositions do not automatically imply sparse pixel-level decompositions after alpha blending. ColorGradedGaussians formalizes this failure explicitly with a two-Gaussian example in which sparse per-primitive palette weights become dense per-pixel after front-to-back accumulation (Chao et al., 2 Apr 2026).

3. Appearance models beyond fixed per-primitive color

A major line of development replaces the original “single view-dependent color per primitive” assumption with richer appearance models. SuperGaussians identifies the limitation directly: in standard Gaussian primitives, any ray that intersects a primitive with the same direction receives the same color regardless of intersection location, which makes the representation non-compact for high-frequency textures (Xu et al., 2024). Its solution is spatially varying color and opacity within a single primitive, implemented through bilinear interpolation, movable kernels, or tiny neural networks on local surfel coordinates $c$8 (Xu et al., 2024).

The movable-kernel formulation defines $c$9 learnable 2D kernels $\Sigma = R S S^{\top} R^{\top},$0 and uses

$\Sigma = R S S^{\top} R^{\top},$1

with

$\Sigma = R S S^{\top} R^{\top},$2

where the paper reports movable kernels as the best-performing spatial variant overall (Xu et al., 2024). ColorGS adopts a closely related screen-space anchor formulation: $\Sigma = R S S^{\top} R^{\top},$3 with $\Sigma = R S S^{\top} R^{\top},$4 and $\Sigma = R S S^{\top} R^{\top},$5 by default (Ji et al., 26 Aug 2025).

Other extensions preserve per-primitive color but alter its semantic role. In GS-Diff, appearance comparison is restricted to the diffuse DC SH coefficient $\Sigma = R S S^{\top} R^{\top},$6, explicitly discarding higher-order SH terms to avoid false positives from view-dependent effects such as reflections and specularity (Galappaththige et al., 8 May 2026). In BiGS, the per-primitive appearance model is generalized from view-only color to a bidirectional light/view scattering representation. Its outgoing radiance is decomposed into direct and indirect terms,

$\Sigma = R S S^{\top} R^{\top},$7

with direct illumination written as

$\Sigma = R S S^{\top} R^{\top},$8

and the directional scattering function represented using bidirectional SH (Liu et al., 2024). This model does not require explicit normals and is intended to unify surface and volumetric materials under dynamic illumination (Liu et al., 2024).

RT-GS introduces another physically motivated extension by separating diffuse, reflection, and transmittance Gaussian primitives. Diffuse Gaussians are rasterized to obtain per-pixel material maps including normal $\Sigma = R S S^{\top} R^{\top},$9, roughness $R$0, base specular reflectance $R$1, and specular blending weight $R$2, while reflection and transmittance Gaussians are ray traced to obtain reflected and transmitted colors $R$3 and $R$4. Final color is then

$R$5

with $R$6 and $R$7 given by a microfacet BRDF/BTDF model (Zeng et al., 1 Apr 2026).

4. Layered and intrinsic decompositions

A particularly explicit reinterpretation of colored Gaussian primitives appears in intrinsic decomposition. “Intrinsic decomposition and editing of 3D Gaussian splats” extends the classic intrinsic image model

$R$8

to Gaussian radiance fields by splitting the representation into three independent Gaussian sets: albedo $R$9, shading $S$0, and view-dependent residual $S$1 (Lanvin et al., 30 Jun 2026). Each set renders its own image,

$S$2

$S$3

$S$4

and the final color is combined at the image level as

$S$5

Crucially, transmittance and alpha are computed independently per layer; there is no cross-layer transmittance coupling (Lanvin et al., 30 Jun 2026).

This decomposition changes the role of a primitive from “a single radiance carrier” to “a layer-specific radiance carrier.” Albedo and shading use constant RGB triplets per splat; residuals use spherical harmonics up to 3 bands with progressive band addition (Lanvin et al., 30 Jun 2026). Geometry is not shared across layers: each set has its own positions and covariances,

$S$6

allowing different spatial resolutions and anisotropies for texture, soft or cast shadows, and view-dependent effects (Lanvin et al., 30 Jun 2026).

A related multi-layer physical separation appears in RT-GS, where the three Gaussian primitive types are diffuse, reflection, and transmittance rather than albedo, shading, and residual (Zeng et al., 1 Apr 2026). BiGS likewise decomposes lighting response into transport and bidirectional scattering components (Liu et al., 2024). These works differ in objective—editing, relighting, or transparency—but share the structural idea that colored Gaussian primitives need not form a single homogeneous field.

5. Optimization, guidance, and editing workflows

Optimization strategies for colored Gaussian primitives vary with the appearance model. In intrinsic decomposition, training proceeds in three phases. First, a video diffusion prior predicts temporally consistent albedo maps $S$7 using DiffusionRenderer with Brick Diffusion; the albedo field $S$8 is optimized with loss

$S$9

using depth and scale regularization (Lanvin et al., 30 Jun 2026). Second, shading splats are optimized with the albedo frozen, using inverse-domain photometric loss and a gray regularization

$x_i \in \mathbb{R}^3$0

to encourage near-achromatic shading (Lanvin et al., 30 Jun 2026). Third, residual splats are spawned where $x_i \in \mathbb{R}^3$1 is high and optimized under the full formation model (Lanvin et al., 30 Jun 2026).

This optimization enables a specific editing workflow: the user selects a rectangular region, dominant albedo splats are identified, a proxy plane is fitted by RANSAC, edited albedo targets are rendered from nearby viewpoints by compositing the textured proxy plane with the albedo field, and $x_i \in \mathbb{R}^3$2 is re-optimized with the edited targets. To capture new high-frequency details, additional albedo splats are spawned on the proxy plane by sampling 20% of pixels covered by the edit (Lanvin et al., 30 Jun 2026). Because shading and residual fields remain untouched, the method preserves lighting and shadows under arbitrary viewpoints (Lanvin et al., 30 Jun 2026).

ColorGradedGaussians offers a different editing paradigm. Instead of directly changing primitive RGB values, it splats view-dependent palette weights into view space, defines chromaticity in CIELAB $x_i \in \mathbb{R}^3$3 space and lightness separately, and enables palette-based recoloring, per-palette tone curves, and pixel-level constraints (Chao et al., 2 Apr 2026). Pixel-level constraints are satisfied by a fast alternating sparse optimization over palette anchors and tone curves while leaving Gaussian parameters fixed; the paper reports convergence in $x_i \in \mathbb{R}^3$4 s (Chao et al., 2 Apr 2026). The distinction is important: this method does not edit primitives by changing their colors directly, but by editing the decomposition through which splatted primitive weights map to chroma and lightness (Chao et al., 2 Apr 2026).

In GLUT, editing is again local but now in RGB space rather than 3D scene space. Given a color constraint $x_i \in \mathbb{R}^3$5, the residual

$x_i \in \mathbb{R}^3$6

is distributed among the $x_i \in \mathbb{R}^3$7 most influential Gaussians at $x_i \in \mathbb{R}^3$8 according to their normalized weights, and only the affine biases $x_i \in \mathbb{R}^3$9 are updated: $s_i \in \mathbb{R}^3$0 Because Gaussian influence decays with Mahalanobis distance in RGB space, edits remain localized without retraining (Xue et al., 19 May 2026).

6. Comparative performance, applications, and technical trade-offs

The literature uses colored Gaussian primitives for several tasks: novel-view synthesis, relighting, transparency and reflection modeling, editing, change detection, surgical scene reconstruction, pruning, and continuous color transformation. Reported results are therefore task-specific rather than directly comparable.

For intrinsic decomposition and editing, the synthetic-scene results in Table 2 show albedo quality of PSNR 29.419, SSIM 0.932, LPIPS 0.095 and full-render quality of PSNR 31.016, SSIM 0.923, LPIPS 0.117, with 2.089M splats and training time 00:27:11. GI-GS attains better full inverse-rendering metrics but its albedo retains shading residuals, with albedo PSNR 13.675, while R3GS fails on the extended indoor scene (Lanvin et al., 30 Jun 2026). The same paper reports that its method adds 1–3× more primitives overall than vanilla 3DGS and is slower in both training and inference because of multi-layer rendering (Lanvin et al., 30 Jun 2026).

For surgical scene reconstruction, ColorGS reports PSNR 39.85, SSIM 97.25\%, and LPIPS 0.03 on EndoNeRF, exceeding Deform3DGS by +1.50 dB PSNR; on StereoMIS it reports PSNR 32.64, SSIM 89.64\%, and LPIPS 0.14 (Ji et al., 26 Aug 2025). Its color-focused ablation shows ColorGS outperforming 3DGS, 2DGS, and SuperGaussian on EndoNeRF, which the paper attributes to improved color fidelity under subtle tissue texture variation and complex illumination (Ji et al., 26 Aug 2025).

For spatially varying intra-primitive models, SuperGaussians reports on Synthetic Blender that Ours-MK achieves PSNR 34.10, SSIM 0.970, LPIPS 0.030, versus 2DGS at PSNR 32.87, SSIM 0.965, LPIPS 0.038. On Mip-NeRF360, Ours-MK reaches PSNR 27.31, SSIM 0.815, LPIPS 0.209; on Tanks and Temples, PSNR 23.72, SSIM 0.847, LPIPS 0.179 (Xu et al., 2024). The paper also reports that movable kernels increase per-primitive parameter count by 1.40× relative to 2DGS, and that rendering on Synthetic Blender decreases from 210.73 FPS for 2DGS to 133.25 FPS for Ours-MK (Xu et al., 2024).

For ray-traced Gaussian primitives, RaySplats reports metrics comparable to 3DGS and 3DGRT while using RGB colors instead of SH, with example values of SSIM $s_i \in \mathbb{R}^3$1, PSNR $s_i \in \mathbb{R}^3$2 on Mip-NeRF360 and SSIM $s_i \in \mathbb{R}^3$3, PSNR $s_i \in \mathbb{R}^3$4 on Deep Blending (Byrski et al., 31 Jan 2025). Stochastic Ray Tracing of Transparent 3D Gaussians emphasizes efficiency on lower-end hardware and reports interactive Windows GPU performance of 76–110 FPS, versus 282–324 FPS for 3DGS rasterization on the tested scenes (Sun et al., 9 Apr 2025). REdiSplats reports SSIM 0.848, PSNR 27.60, LPIPS 0.234 on Mip-NeRF360; SSIM 0.822, PSNR 22.17, LPIPS 0.21 on Tanks and Temples; and SSIM 0.911, PSNR 30.01, LPIPS 0.316 on Deep Blending (Byrski et al., 15 Mar 2025). RT-GS reports best PSNR on Sedan (26.203) and Spheres (23.050) for reflections, and on transparency achieves best PSNR/SSIM/LPIPS on PopcornCup (25.720/0.874/0.199) and TallBottle (26.225/0.888/0.172) (Zeng et al., 1 Apr 2026).

In primitive-space change detection, GS-Diff uses only native primitive attributes $s_i \in \mathbb{R}^3$5, compares DC colors rather than full SH appearance, and reports mIoU 0.644 and F1 0.758 on PASLCD, surpassing the strongest prior image-space baseline O-SCD at mIoU 0.552 and F1 0.694, with an improvement of approximately 17% in mIoU (Galappaththige et al., 8 May 2026). This application treats colored Gaussian primitives not primarily as renderable appearance units, but as comparable 3D entities whose geometry and color can be scored separately (Galappaththige et al., 8 May 2026).

GLUT occupies a different application domain but offers an informative contrast. With $s_i \in \mathbb{R}^3$6 Gaussian primitives in RGB space, GLUT-64 reports PSNR 48.42 and $s_i \in \mathbb{R}^3$7 on 75 LUT Hald images, and PSNR 47.90, SSIM 0.998, LPIPS 0.002, $s_i \in \mathbb{R}^3$8 on natural images. It requires 1,420 parameters, versus grid 3D LUT memory of $s_i \in \mathbb{R}^3$9 MB at $p \in \mathbb{R}^3$00, and reaches 176 FPS at 4K on a 4090 GPU (Xue et al., 19 May 2026). This suggests that Gaussian-localized appearance primitives can be computationally attractive even outside scene rendering.

7. Limitations, misconceptions, and open directions

Several recurring limitations are explicit in the literature. Intrinsic decomposition depends on diffusion-predicted albedo maps that can be blurry or mildly nonlinear, causing high-frequency detail to leak into shading and mild color shifts; it is also not a full inverse-rendering system and does not estimate illumination, so relighting remains out of scope (Lanvin et al., 30 Jun 2026). RT-GS assumes infinitely thin transparent shells and does not model Snell refraction, absorption, multiple internal reflections, or complex multi-layer glass (Zeng et al., 1 Apr 2026). RaySplats notes that extremely flat Gaussians can challenge intersection stability and that its Gaussian shading model does not define normals or BRDFs for primitives (Byrski et al., 31 Jan 2025). REdiSplats similarly notes that highly glossy or microfacet effects may not be fully captured by per-primitive color alone in its dedicated renderer (Byrski et al., 15 Mar 2025).

A common misconception is that “colored Gaussian primitives” necessarily means only per-primitive RGB or SH color. The recent literature shows at least five meanings: RGB-colored volumetric Gaussians for direct ray tracing (Byrski et al., 31 Jan 2025); spatially varying intra-primitive color and opacity fields (Xu et al., 2024, Ji et al., 26 Aug 2025); layered intrinsic radiance fields for albedo, shading, and residual (Lanvin et al., 30 Jun 2026); physically typed reflection and transmittance primitives (Zeng et al., 1 Apr 2026); and Gaussian-localized affine transforms in RGB space (Xue et al., 19 May 2026). This suggests that the term now denotes a family of Gaussian primitives whose appearance payload is explicit and semantically meaningful, rather than a single fixed parameterization.

Another misconception is that richer primitive appearance always improves editing locality. ColorGradedGaussians shows that sparse primitive-space palette weights can still yield dense pixel-space decompositions after alpha blending, degrading edit specificity (Chao et al., 2 Apr 2026). SafeguardGS makes an analogous point from the pruning side: primitive importance depends not only on alpha contribution but also on color similarity to the target pixel, and pruning that ignores color can destroy high-frequency chromatic detail (Lee et al., 2024).

Current directions indicate two broad trajectories. One is increasing physical structure: bidirectional transport in BiGS, microfacet reflection and transmittance in RT-GS, and ray-traced ellipsoids or mesh-backed flat Gaussians in RaySplats and REdiSplats (Liu et al., 2024, Zeng et al., 1 Apr 2026, Byrski et al., 31 Jan 2025, Byrski et al., 15 Mar 2025). The other is increasing controllability and decomposition: intrinsic albedo-shading-residual fields, view-space palette decompositions, and localized Gaussian editing in RGB space (Lanvin et al., 30 Jun 2026, Chao et al., 2 Apr 2026, Xue et al., 19 May 2026). A plausible implication is that future work will continue to split the traditional single appearance channel of 3DGS into multiple explicit subspaces—geometric, photometric, material, or colorimetric—while preserving the efficiency advantages that made Gaussian splatting attractive in the first place.