ColorGradedGaussians: Palette-Based Color Grading for 3D Gaussian Splatting via View-Space Sparse Decomposition

Abstract: Professional color editing requires precise control over both color (hue and saturation) and lightness, ideally through separate, independent controls. We present a real-time interactive color editing framework for 3D Gaussian Splatting (3DGS) that enables palette-based recoloring, per-palette tone curves for color-aware lightness adjustment, and accurate pixel-level constraints -- capabilities unavailable in prior palette-based 3DGS methods. Existing approaches decompose colors at the primitive level, optimizing per-Gaussian palette weights before splatting. However, sparse primitive-level weights do not guarantee sparse pixel-level decompositions after alpha-blending, causing palette edits to affect unintended regions and degrading editing quality. We address this through view-space palette decomposition, splatting weights instead of colors to optimize the observable appearance of the scene. We introduce a geometric loss using inverse barycentric coordinates to enforce consistent sparsity patterns, ensuring similar colors share similar decompositions. Our approach achieves superior editing quality compared to primitive-space methods, enabling professional color grading workflows for 3DGS scenes with real-time interaction.

• The paper introduces a novel view-space palette decomposition that enables real-time, localized editing of 3D Gaussian Splatting scenes.
• It leverages spherical harmonics and geometric consistency loss to enforce sparsity and maintain palette compactness and color fidelity.
• Experimental results show improved PSNR, SSIM, and LPIPS compared to primitive-space methods, delivering superior edit locality.

Real-Time Palette-Based Color Grading for 3D Gaussian Splatting: View-Space Sparse Decomposition

Motivation and Background

This work introduces ColorGradedGaussians, a novel method that enables real-time, multi-modal color grading of 3D Gaussian Splatting (3DGS) scenes using a palette-based view-space sparse decomposition. Artistic and professional color grading workflows demand independent and selective control over chroma and lightness—requirements unmet by prior palette-based approaches for 3DGS, which operate at the primitive (per-Gaussian) level. Such prior methods, including PaletteGaussian and RecolorGaussian, decompose each Gaussian’s color as a mixture of global palette colors, but alpha-blending during rendering breaks sparsity, propagates edits to unintended regions, and makes lightness manipulation inextricable from chroma editing.

ColorGradedGaussians addresses these deficiencies by shifting palette decomposition from primitive space to view space: instead of splatting colors, the method splats palette weights, enabling pixel-level sparsity and direct regularization where edits are observed and specified. This approach allows for three essential editing modalities: direct palette manipulation, per-palette tone curves for color-aware lightness control, and pixel-level constraints for localized color targets—all with consistent propagation to novel views and real-time responsiveness.

View-Space Palette Decomposition and Architecture

The proposed system decomposes 3DGS rendering into palette-based chromaticity and lightness components within CIELAB color space, leveraging spherical harmonics (SH) for view-dependent representation of both palette weights and lightness. Each Gaussian parameterizes view-dependent palette weights via SH expansion, which are rasterized (splatting) and normalized with softmax to produce interpretable, per-pixel palette weights.

Lightness is encoded separately, again via SH splatting and sigmoid normalization, so chromaticity and lightness are independently editable. A fixed grey palette vertex ensures achromatic coverage, with $K-1$ trainable chromatic palette vertices forming a compact, representative palette in normalized ab-space.

Figure 1: The training pipeline for ColorGradedGaussians utilizes direct supervision of splatted palette weights via geometric loss and ensures palette compactness.

Sparse Decomposition and Regularization

Sparsity and edit locality are enforced via a geometric consistency loss based on differentiable inverse barycentric coordinates. Target weights are computed as soft barycentric coordinates of each pixel’s ground truth chromaticity in the palette’s tessellated ab-space. Minimizing the distance between splatted and geometric target weights drives consistent, localized decompositions among perceptually similar colors. Sparsity is further encouraged through an adapted Aksoy sparsity metric, with careful exclusion of the grey component to avoid degenerate solutions.

Palette compactness and representativity are maintained by penalizing grey dominance and enforcing minimum distance among chromatic palette vertices. This balancing of losses prevents palette expansion beyond the gamut or clustering of vertices, both of which would compromise editability.

Figure 2: Geometric sparsity regularization ensures consistent palette weights and prevents edit bleeding across semantic regions.

Figure 3: Grey penalization ablation demonstrates how palette compactness and chromatic representativity are maintained for high editability.

Real-Time Editing Modalities

ColorGradedGaussians enables three principal editing modalities:

• Palette Manipulation: Direct adjustment of palette vertices in ab-space instantly propagates chromatic edits throughout the scene according to pixel weights.
• Per-Palette Tone Curves: Independent tone curves for each palette color allow chroma-specific lightness and contrast manipulation. Tone curve control points are interpolated with biharmonic functions under natural boundary conditions, facilitating professional-grade lightness adjustment.
• Pixel-Level Constraints: Users specify color targets on individual pixels in any view, and a fast alternating optimization uniquely adjusts palette and tone curves. Because the decomposition happens in view space, only the small set of parameters (palette and curves) need to be optimized, yielding real-time interactive results without re-optimizing Gaussian or SH parameters.

  Figure 4: Independent lightness control per palette color demonstrates chroma-aware tone curve editing.

  

  Figure 5: Real-time pixel-level editing constrains colors on reference views, propagating local or global edits across novel views.

Comparative Analysis and Numerical Findings

ColorGradedGaussians achieves superior editing locality and consistency, as evidenced by quantitative and qualitative comparisons against PaletteGaussian and RecolorGaussian. The method preserves reconstruction fidelity, with PSNR of 24.52, SSIM of 0.82, and LPIPS of 0.14, outperforming previous approaches on held-out test views.

Primitive-space methods fail to produce sparse, consistent pixel-level palette weights, leading to color bleeding and unpredictable edit propagation. In contrast, ColorGradedGaussians’ view-space geometric loss enforces consistent decompositions among similar colors, yielding highly localized edits and superior edit quality. Real-time performance is maintained, with interactive editing optimizations completing in approximately 0.02 seconds.

Figure 6: Editing sparsity analysis demonstrates superior locality and reduced edit bleeding with the proposed view-space approach.

Figure 7: Pixel-level editing comparison highlights real-time, localized edits achieved by ColorGradedGaussians, versus bleeding and slow convergence in primitive-space methods.

Implications, Limitations, and Future Directions

Practically, ColorGradedGaussians enables professional color grading workflows for 3DGS, including independent lightness control, chromatic recoloring, and localized edits, all consistent across novel views and performed in real time. This methodological advance bridges the gap between 2D palette-based recoloring and multi-view consistent editing in high-fidelity 3D scene representations.

Theoretical implications include the demonstration that palette-based decomposition must occur in view space to maintain edit locality and sparsity after non-linear blending. The geometric consistency loss, by providing differentiable soft barycentric assignments, ensures stable palette optimization and robust edit propagation.

Limitations include increased storage due to SH expansion for view-dependent weights (approximately $2\times$ that of standard 3DGS for typical palette sizes), and the lack of object-level or semantic editing constraints. Future directions may involve more compact directional representations (spherical Gaussians, spherical beta, Voronoi partitions), RGBA support, and integration of segmentation-aware palette decompositions to enable selective object/material recoloring while maintaining locality.

Conclusion

ColorGradedGaussians establishes view-space palette-based decomposition as the requisite approach for high-quality, real-time color grading in 3DGS. The method achieves multi-modal editing—including chroma, lightness, and pixel-level constraints—with superior locality and consistency while preserving reconstruction fidelity and real-time performance. The rigorous geometric and sparsity regularization framework ensures palette representativity and editability. Extensions to more efficient representations and semantic editing will further enhance applications in professional content creation and scene manipulation.

(2604.01551)