Differentiable Splatting & Compositing

Updated 23 June 2026

Differentiable splatting and compositing are techniques that render parametric primitives using smooth, spatially extended kernels for seamless forward and backward propagation.
They aggregate the influences of multiple primitives via analytic alpha compositing, enabling effective optimization of geometry, appearance, and topology.
These methods drive applications in vector graphics, 3D reconstruction, inverse rendering, and neural scene optimization by improving efficiency and rendering fidelity.

Differentiable splatting and compositing refers to a family of techniques for rasterizing parametric primitives (e.g., Bézier curves, edges, 3D Gaussians, surfaces, triangles, or neural fields) onto discrete image grids using spatially extended kernel functions (typically Gaussians or soft-windowed indicators), such that the forward rendering and backward gradient computation are smoothly defined with respect to all primitive parameters. This has enabled efficient end-to-end optimization of geometry, appearance, and topology from multi-view images, vector graphics, or scene representations, with broad adoption in inverse rendering, neural scene reconstruction, and differentiable graphics workflows.

1. Mathematical Foundations and Generic Formulation

Differentiable splatting begins by defining a primitive—often a point, curve, surface element, or explicit mesh—that is mapped (or "splatted") to the image domain via a kernel function $K(\cdot)$ centered at a location determined by the primitive's parameters (e.g., pose, shape, orientation). Common choices include isotropic or anisotropic Gaussians, soft triangles, or learned window functions. The compositing pipeline aggregates multiple primitives’ image-plane influences, typically using alpha compositing (over-operator) or weighted sums, applied in depth order to handle occlusion.

For $N$ primitives, each associated with an image-space kernel $K_i(x)$ , color $c_i$ , and opacity $\alpha_i$ , the compositing equation at pixel $x$ is:

$I(x) = \sum_{i=1}^N c_i \, \alpha_i(x) \prod_{j < i} (1 - \alpha_j(x)),$

where $\alpha_i(x)$ = kernel response modulated by learned opacity (e.g., $G_{i}(x) = \exp( -\frac{1}{2}(x-\mu_i)^T\Sigma_i^{-1}(x-\mu_i))$ , $\alpha_i(x) = o_i \, G_i(x)$ ) (Ying et al., 18 Mar 2025, Liu et al., 20 Mar 2025, Tran et al., 10 Mar 2026).

All steps in kernel evaluation and compositing are constructed from analytic, differentiable functions, enabling backpropagation of losses from image space (e.g., photometric, feature, or silhouette losses) to geometric and appearance parameters. Chain-rule gradients are computed for kernel center, covariance (shape), rotation, scale, learned color/appearance, and compositing order (Held et al., 25 May 2025, Younes et al., 7 May 2026, Guo et al., 24 Jan 2026).

2. Families of Differentiable Splatting and Compositing Techniques

2D Vector Graphics (Bézier splatting): Parametric Bézier curves are sampled to create a set of 2D Gaussians (with means on the curve and covariances encoding stroke orientation/width), which are splatted and composited with alpha blending (Liu et al., 20 Mar 2025). Analytic gradients with respect to Bézier control points, color/stroke, and kernel parameters enable high-performance, high-fidelity differentiable vector graphics rendering.
Edge Sketch Splatting: SketchSplat represents 3D parametric edges as line or Bézier primitives with control points, opacity, and scale. Points sampled along 3D sketches are projected with camera intrinsics/extrinsics and splatted as 2D Gaussians onto edge images (Ying et al., 18 Mar 2025). Topological adaptive operations (endpoint/overlap/colinear merging, visibility pruning) are interleaved with differentiable optimization to maintain edge compactness and structural accuracy.
3D Gaussian Splatting (3DGS): Photo-realistic 3D scene representation is achieved using anisotropic 3D Gaussians with learned center, covariance, color, and opacity. Image- or depth-space projection followed by elliptical Gaussian splatting and front-to-back alpha compositing yields fast, differentiable, high-fidelity rendering (Tran et al., 10 Mar 2026, Guo et al., 24 Jan 2026, Yuan et al., 24 May 2025).
Triangle and Mesh Splatting: Triangle Splatting and its extensions (including 2D Triangle Splatting, Triangle Splatting+) treat each triangle as a “soft” kernel in the image plane, defined through signed-distance fields normalized at the incenter, powered by a sharpness parameter (Held et al., 25 May 2025, Held et al., 29 Sep 2025, Sheng et al., 23 Jun 2025). Differentiable depth sorting and volume-rendering-style compositing propagate gradients to triangle vertices, colors, and opacities. Mesh connectivity may be enforced via shared vertex pools.
Surface and Fourier Splatting: 3DSS replaces volumetric Gaussian groups with oriented surfels, using direct coverage-based compositing and Shepard normalization, enabling expressive silhouettes and robust gradients at edges (Younes et al., 7 May 2026). Fourier Splatting parameterizes primitive shapes using Fourier boundary encoding, with sharp power-window compositing and run-time level-of-detail selection (Jurca et al., 20 Mar 2026).
Object-Centric Neural Fields (MLP-Splatting): Each primitive is associated with a compact neural field predicting radiance and opacity, spatially masked by an anisotropic Gaussian window. Sparse volumetric compositing is performed at the primitive–ray soft-contact point; gradient flow supports object-centric decomposition under RGB supervision (Kim et al., 2 Jun 2026).

Table: Core Splatting Primitives and Compositing Schemes

Method	Primitive	Kernel/Window	Compositing
Bézier Splatting	Bézier curve	2D Gaussian	Alpha-blending (over)
SketchSplat	3D line/Bézier edge	2D Gaussian (per sample)	Sum or alpha-blending
3DGS	3D Gaussian	2D elliptical Gaussian	Alpha-blending (depth-ordered)
Triangle Splatting	3D triangle (mesh)	SDF-based window	Volume-like alpha-blending
3DSS	Oriented surfel	Gaussian, coverage-based	Front-to-back, per-layer
Fourier Splatting	Planar surfel, Fourier-bound	Power-window, $N$ 0	Alpha-blending
MLP-Splatting	Local neural primitive	Gaussian window	Sparse volumetric

3. Differentiable Compositing Principles and Backpropagation

The central compositing operator is the associatively and differentiably defined front-to-back alpha blending:

$N$ 1

All analytic functions, including exponentials, products, and cumulative sums, are differentiable. This supports direct gradient backpropagation from image losses to primitive parameters (control point positions, scales, orientation/rotation, opacity, neural field weights). For example, the gradients with respect to a Gaussian’s center, covariance, or color are analytic:

$N$ 2
$N$ 3, where $N$ 4 is the blending weight at that depth.

This analytic gradient flow is essential for efficient autograd in deep learning frameworks and high-throughput GPU renderers (Yuan et al., 24 May 2025, Guo et al., 24 Jan 2026).

For variance-aware and uncertainty-aware splatting, the law of total variance is combined with alpha compositing, enabling differentiable rendering of per-pixel uncertainty maps (e.g., VarSplat) (Tran et al., 10 Mar 2026). Indirect effects, such as view-dependent coverage, can be handled by defining additional objectives and regularizers that propagate gradients through the entire rendering pipeline.

Effective differentiable splatting frameworks incorporate periodic adaptive operations to manage scene complexity, ensure compactness, and maintain local detail:

Pruning: Remove “dead” or redundant primitives (e.g., curves of low average opacity or small area, triangles with negligible blending weight, surfels with low gradients), often based on dynamically decaying thresholds (Liu et al., 20 Mar 2025, Held et al., 29 Sep 2025, Held et al., 25 May 2025, Younes et al., 7 May 2026).
Densification/Splitting: Spawn new primitives in regions with high reconstruction error or insufficient coverage, e.g., new Bézier curves in high-error regions (Liu et al., 20 Mar 2025); four-way triangle subdivision (Held et al., 25 May 2025); MCMC and learned-lobe splits for Fourier surfels (Jurca et al., 20 Mar 2026).
Merging/Topology Control: Merge colinear or overlapping sketches (edges or lines), collapse triangle endpoints, or enforce mesh connectivity by pooling shared vertices (Ying et al., 18 Mar 2025, Held et al., 29 Sep 2025).

This dynamic adaptation enables models to escape local minima, efficiently allocate capacity, and produce high-fidelity, compact representations.

5. Hardware, Software, and Implementation Performance

Differentiable splatting and compositing has been implemented in both CPU/GPU tile-based and hardware-rasterization pipelines, leveraging the associativity and locality of the core splatting/compositing operators:

PyTorch and autodiff frameworks are used for rapid prototyping, with performance up to $N$ 5 faster than pixel-wise rasterization (e.g., Bézier Splatting vs. DiffVG for vector graphics) (Liu et al., 20 Mar 2025).
GPU hardware rasterizers exploiting programmable blending achieve $N$ 6 to $N$ 7 speedup in backward-pass rasterization and over $N$ 8 lower memory usage compared to tile-based approaches, especially when using 16-bit render targets (Yuan et al., 24 May 2025). Forward-replay caches enable unrolled compositing with minimal bandwidth under on-device constraints (Guo et al., 24 Jan 2026).
Sparse compositing and tiling: Only primitives overlapping a tile need to be evaluated (MLP-Splatting, Triangle Splatting), yielding $N$ 9 the memory and $K_i(x)$ 0 the speed for semantic scene editing (Kim et al., 2 Jun 2026). Anti-aliasing and MIP prefiltering are supported via analytic or Jacobian-mapped filtering, as in 3DSS (Younes et al., 7 May 2026).
Export and compatibility: Learned parametric primitives remain compatible with standard mesh, SVG, or graphics pipelines, facilitating downstream rendering, relighting, and editing in real-time engines (Held et al., 29 Sep 2025, Liu et al., 20 Mar 2025).

6. Applications, Empirical Results, and Limitations

Differentiable splatting and compositing methods deliver broad empirical success:

Vector graphics: Bézier Splatting produces higher SSIM and PSNR than DiffVG, converges $K_i(x)$ 1 faster, and supports SVG export (Liu et al., 20 Mar 2025).
3D reconstruction and view synthesis: Triangle Splatting, 3DGS, and MLP-Splatting achieve competitive or superior fidelity to NeRF, with real-time rendering and object-level semantic control (Held et al., 25 May 2025, Kim et al., 2 Jun 2026, Jurca et al., 20 Mar 2026).
Physics-driven 4D tracking: PersistGS embeds a differentiable rigid-body simulator in the splatting loop to enforce object permanence through occlusion, outperforming generative extrapolation and hallucination priors (Ramlal et al., 2 Jun 2026).
Inverse rendering: 3DSS jointly optimizes shape, SVBRDF, and illumination, matches or exceeds mesh-based and volumetric baselines across view synthesis and relighting (Younes et al., 7 May 2026).

Current limitations include:

Non-manifoldness or “ghost” artifacts (floating primitives) in weakly supervised regions, mitigated by pruning and regularization (Held et al., 25 May 2025, Younes et al., 7 May 2026).
Mesh connectivity is not always preserved during training (except in methods that tie vertex updates across triangles) (Held et al., 29 Sep 2025).
Certain kinks (e.g., SDF max-operators for triangle windows) introduce minor non-smoothness, but practical autodiff handles these without bias or instability.

7. Comparative Perspectives and Scalability

A major advantage of differentiable splatting/compositing is the native trade-off between explicitness (direct geometry, topology), neural expressivity (MLP, Fourier, or spherical harmonics on primitive attributes), memory/compute efficiency, and real-time compatibility. Fourier Splatting and LoD truncation methods provide fine-grained scalable fidelity control, enabling bandwidth-constrained or hierarchical rendering without re-optimization (Jurca et al., 20 Mar 2026). Compared to dense radiance fields, splatting-based methods require orders of magnitude fewer primitives, natively support semantic or part decomposition (MLP-Splatting), and are amenable to direct mesh conversion and real-time simulation.

These advances position differentiable splatting and compositing as foundational tools across neural graphics, inverse rendering, SLAM, and vector graphics, unifying classical computer graphics and modern differentiable learning paradigms.