Point Splatting: Methods & Applications
- Point splatting is a rendering technique where discrete 3D or 2D points are assigned finite spatial support, projected into a target domain, and composited to form continuous signals, enhancing applications ranging from LiDAR simulation to novel-view synthesis.
- It overcomes discontinuities and aliasing in traditional point rendering by using anisotropic isotropic Gaussian kernels, surfels, or learned ellipsoids, providing flexibility in geometric and appearance representations.
- Key methods include Differentiable Surface Splatting (DSS) for geometry optimization, Gaussian splatting for efficient GPU rasterization, and advanced systems like PointGS and TriPS for multi-view consistent visual rendering.
- Follow-up_questions: Key Follow Up Questions
Point splatting is a family of explicit rendering and inverse-rendering methods in which discrete 3D or 2D points are assigned finite spatial support, projected or aggregated into a target domain, and composited or averaged to form continuous signals. Depending on the representation, a splat may be a surfel, oriented disk, isotropic or anisotropic Gaussian, learned ellipsoid, or a kernel-generated image point. Point splatting is used for geometric processing, novel-view synthesis, LiDAR and radar simulation, point-cloud upsampling, radiomap extrapolation, segmentation, registration, and differentiable optimization. Its principal purpose is to overcome the discontinuities, holes, aliasing, and weak gradients associated with rendering points as isolated pixels while retaining the computational advantages of explicit point-based representations.
1. Conceptual foundations and primitive representations
Point splatting differs from naïve point rendering because a point is not treated solely as a sample to be covered by one pixel. It is assigned a finite footprint whose contribution varies over neighboring pixels, voxels, grid cells, range bins, Doppler bins, or other target-domain elements. The footprint may be fixed or learned, isotropic or anisotropic, surface-based or volumetric, and may carry geometric, appearance, semantic, or physical-scattering attributes.
A conventional surface-splatting primitive is a surfel, a local surface element carrying at least a position, normal, and appearance. A surfel is projected to a finite screen-space footprint, commonly an elliptical disk or Gaussian derived from its local tangent plane and camera projection. Its contribution is filtered and composited with other projected surfels. The corresponding pipeline consists of projecting the point or surfel, projecting its local tangent-plane footprint, evaluating a disc or Gaussian kernel, computing surface-consistent depth, resolving visibility, and compositing contributions. Differentiable Surface Splatting (DSS) formalizes this approach for point-cloud geometry processing (Yifan et al., 2019).
DSS uses surface-aware rather than purely point-centered rendering. Its primitive is a projected local surface patch, not merely a point assigned a screen-space radius. The method optimizes point positions and normals and introduces projection and repulsion regularization to keep points near an underlying surface and discourage clustering. Its optimization objective is schematically
where is image-rendering loss, is the projection regularizer, and is the repulsion regularizer. The method is designed to avoid explicit mesh connectivity and to accommodate both large topology changes and local geometric detail.
A Gaussian splat replaces the explicit surface disk with an anisotropic spatial kernel. A typical Gaussian primitive has mean , covariance , opacity , and appearance attributes. Its covariance is commonly parameterized as
where is a diagonal scale matrix and is a rotation derived from a quaternion. The covariance determines orientation and spatial extent, while opacity controls visibility-weighted contribution. In 3D Gaussian Splatting, color may be represented by spherical-harmonic coefficients, allowing view-dependent appearance.
The principal representation families can be summarized as follows.
| Primitive | Main attributes | Typical target domain |
|---|---|---|
| Surfel | Position, normal, appearance, radius | Camera image |
| Oriented disk | Center, normal, radius, metadata | LiDAR rays and geometry |
| Anisotropic Gaussian | Mean, covariance, opacity, appearance | Camera image or feature maps |
| Learned ellipsoid | Position correction, scale, rotation, opacity, color | Point-cloud rendering |
| Radar point | Position, area, normal, material, velocity | Range, angle, Doppler |
The term “Point Splatting” is therefore broader than 3D Gaussian Splatting. It denotes a rendering principle rather than one fixed primitive parameterization.
2. Projection, kernels, visibility, and differentiability
For camera-based rendering, a 3D primitive is transformed into camera coordinates and projected into the image plane. A Gaussian’s projected covariance is commonly approximated through the Jacobian of the local perspective projection:
where is the camera transformation. The projected primitive produces an elliptical image-space footprint. A pixel-dependent opacity can be written as
where is primitive opacity and is the projected Gaussian.
Depth ordering is generally handled by sorting projected contributions from front to back. With effective per-pixel opacity and color , front-to-back compositing is
with transmittance
This formulation provides soft visibility and differentiable dependence on positions, scales, rotations, opacity, and appearance. Tile-based rasterizers assign projected primitives to image tiles, sort them by depth, and evaluate only the relevant footprints.
Differentiability remains difficult because coverage, visibility, ordering, kernel support, and merging can change discontinuously. A projected footprint may begin or cease to cover a pixel; two primitives may exchange depth order; a truncated kernel may cross its cutoff; or a normal change may alter both the projected footprint and surface depth. DSS addresses this problem with a backward-rendering procedure that caches up to projected points per pixel and revisits candidate points, using in the supplied implementation details (Yifan et al., 2019). The exact discontinuity-aware gradient derivation is part of the omitted main text.
Point splatting is useful for optimization because finite kernels provide nonzero gradients over neighborhoods rather than only at exact pixel centers. In fractal inversion, chaotic 2D point trajectories are rendered as isotropic 2D Gaussians with standard alpha blending. The renderer is tile-based, differentiable, uses supersampling, and does not require primitive sorting because the points are two-dimensional (Djeacoumar et al., 24 Feb 2025). The image is rendered at five times the target resolution in each spatial dimension and averaged down; the reported supersampled Gaussian standard deviation is $2.5$ pixels.
The same principle applies to pose refinement. ReFineR assigns each point an isotropic Gaussian whose standard deviation is proportional to nearest-neighbor distance, with point-size multiplier and fixed opacity $o=0.2. The splatted point-cloud channels are compared with image channels through the Gradient Orientation Measure, enabling gradients to flow from pixel-level edge alignment to the six camera-pose degrees of freedom (Insalaco et al., 12 Nov 2026).
3. Geometry processing, point distribution, and LiDAR simulation
Point splatting is particularly useful when explicit mesh connectivity is unavailable, unreliable, or undesirable. DSS optimizes point positions and normals under multi-view image supervision and uses projection and repulsion regularization to maintain a useful point distribution. Excessive repulsion can drive points away from the surface, while excessive projection regularization can prevent topology changes. The supplied appendix reports position and normal learning rates of $5$ and $5000$, respectively, and uses twelve camera views per optimization step (Yifan et al., 2019).
AdaSplats extends surface-aware splatting to large, noisy, nonuniform outdoor LiDAR point clouds (Richa et al., 2022). Its primitive is an oriented disk,
where is the center, the unit normal, and the radius. Basic splat construction estimates local neighborhoods using PCA, takes the least-variance eigenvector as the normal, reorients the normal toward the LiDAR sensor, and grows a splat while point-to-plane error remains below a global threshold. Points within a fraction of a splat radius are discarded from later generation.
The adaptive variant divides points into ground, surface, linear, and non-surface groups. Ground uses larger neighborhoods, while linear and non-surface structures use smaller neighborhoods and stricter point-to-plane tolerances. Candidate inclusion additionally requires semantic consistency and a normal smoothness condition,
This prevents large ground or façade splats from crossing poles, fences, vegetation, and sharp boundaries. AdaSplats-Descr replaces semantic labels with local PCA descriptors for linearity, planarity, and sphericity.
AdaSplats also performs denoising and resampling. Points with point-to-plane error greater than are rejected. When local splat density is below the average, neighboring splats with compatible semantic groups and normals are used to insert midpoint samples. On PC3D-Paris, resampling reduced the number of primitives from $2.84$ million to $1.72$ million, increased simulation rate from $180$ Hz to $205$ Hz, and reduced C2C error from $1.99$ cm to $1.97$ cm (Richa et al., 2022).
For LiDAR simulation, each oriented disk is intersected with a sensor ray. A ray with origin and direction intersects the splat plane at
and the intersection is accepted when it lies within the disk radius. AdaSplats constructs an OptiX BVH from disk axis-aligned bounding boxes, performs GPU ray generation and traversal, and applies exact CUDA ray–plane and disk-radius tests. The simulator models Velodyne HDL-32 and HDL-64 firing sequences, and SimKITTI32 supports up to overlapping intersections, semantic consistency checks, frame-number filtering, and additive white Gaussian range noise with standard deviation $0.005$ (Richa et al., 2022).
4. Learned Gaussian and feature splatting for visual rendering
Modern learned point-splatting systems use point clouds as geometric scaffolds and predict Gaussian attributes or feature representations from them. Unlike per-scene optimized 3DGS, these methods can be feed-forward, generalizable, or specialized for sparse inputs.
TRIPS uses trilinear point splatting for real-time radiance-field rendering (Franke et al., 2024). A point has position, world-space size, learned descriptors, and transparency. Its projected size is
where is focal length, world-space size, and camera-space depth. Points are written into two layers of a screen-space image pyramid selected according to projected size. Each primitive produces a constant-size write, consisting of bilinear interpolation within layers and linear interpolation between layers. A lightweight decoder reconstructs a full-resolution image from the feature pyramid. The representation is fully differentiable with respect to point positions, sizes, features, opacity, camera parameters, and tone-mapping parameters. The reported system maintains a real-time rate of 60 FPS and reaches approximately 11.1 ms per frame in the Playground experiment.
PFGS, or Point Feature Gaussian Splatting, predicts Gaussian rotation, anisotropic scale, and opacity for each input point using a point-wise regressor (Wang et al., 2024). Input colors remain attached to the original points, while a multiscale point feature extractor supplies descriptors. The system performs two rasterizations with shared Gaussian geometry: one for RGB and one for learned feature descriptors. The feature-splat image is concatenated with the RGB-splat image and passed to a progressive multiscale decoder. Its two-stage training first makes Gaussian rendering geometrically and photometrically meaningful and then jointly trains the decoder. On ScanNet, DTU, and THuman2.0, the reported PFGS results are respectively $19.86/0.758/0.452$, $25.44/0.901/0.164$, and $34.74/0.983/0.009$ in PSNR, SSIM, and LPIPS.
Learned splatting can also operate directly on arbitrary point-cloud streams. The Point-to-Ellipsoid Network, P2ENet, predicts a center offset, standard deviations, quaternion, opacity, and normal for every input colored point (Hu et al., 2024). The center is , and the covariance is constructed from predicted scale and rotation. Sparse 3D convolutions in Minkowski Engine provide local and global context. The resulting Gaussians are rasterized with depth sorting and alpha compositing. At , the method reports $33.8$ dB PSNR and $0.9952$ MS-SSIM for compact point clouds, with less than $1$ ms rendering after approximately $27$ ms preprocessing.
PointGS targets sparse-view novel-view synthesis (Xiang et al., 12 Jun 2025). VGGT supplies dense geometry and camera poses; FPN features are sampled at projected point locations; variance aggregation captures inter-view disagreement; and local KNN self-attention refines point descriptors. Two MLPs decode color, opacity, scale, and rotation from coordinates, viewing direction, and enriched features. On LLFF with three views at $1/8$ resolution, PointGS reports $20.54$ PSNR, $0.72$ SSIM, $0.25$ LPIPS, and 413 FPS. Removing point interaction reduces performance to $20.03$ PSNR, while replacing variance aggregation with averaging reduces it to $19.95$ PSNR.
PointSplat applies the same 3D-space principle to compact human reconstruction (Guo et al., 30 Jun 2026). It constructs a visual-hull-like point proxy, voxelizes it, uses ray casting to remove interior points and establish explicit 2D–3D correspondences, and applies a Point-Image Transformer. Each retained point predicts Gaussians. On DNA-Rendering, PointSplat reports $27.18$ PSNR, $0.891$ SSIM, and $0.071$ LPIPS using approximately $71$k Gaussians and $0.4$ seconds of inference. The method’s compactness derives from predicting once per shared 3D point rather than generating redundant per-view Gaussian maps.
LoCoSplat adopts a more minimal form of point splatting (Wang et al., 3 Oct 2026). It unprojects image pixels into 3D anchors, projects 132-dimensional point features into 16-dimensional vectors, and averages them in fine and coarse grids. The coarse grid has cell size , while the fine voxel size is . A pointwise MLP decodes Gaussian parameters from the anchor feature and local averages. The internal operation is therefore point-to-grid feature aggregation rather than final image rendering. On RealEstate10K, it reports $31.51$, $31.47$, and $30.57$ PSNR for 6, 12, and 24 views, respectively, while using substantially less inference memory than VolSplat.
5. Reconstruction, upsampling, segmentation, and registration
Point splatting can be incorporated into reconstruction systems whose output is an explicit point cloud rather than only an image.
GaussianPU uses 3D Gaussian Splatting as a bridge between sparse colored point clouds, restored 2D images, and dense 3D point clouds (Guo et al., 2024). It renders the sparse cloud at point sizes $1$ and , restores the two images with a dual-scale network, and initializes exactly Gaussian primitives through interpolation. Gaussian cloning, splitting, and pruning are disabled; opacity is fixed to one; and Gaussian scale is constrained to a common value. The optimized Gaussian centers and colors form the explicit output cloud. For upsampling, GaussianPU reports $23.684$ PSNR, $0.932$ SSIM, $0.043$ LPIPS, and $0.876$ IW-SSIM, improving over the sparse input and GradPU in the reported WPC evaluation.
PointGauss uses Gaussian primitives as a point-cloud-like domain for multi-object segmentation (Sun et al., 1 Aug 2025). A user click is lifted into 3D by ray casting until accumulated Gaussian opacity exceeds $0.9$. Gaussian-wise prompt weights are computed from distance to the lifted click, and a cylindrical region with radius and height $3.0$ m is processed by PointTransformerV3. The resulting foreground/background labels are stored directly on Gaussian primitives and rendered from new views using GPU label projection. Its key source of multi-view consistency is the shared 3D label field rather than an explicit cross-view loss. On DesktopObjects-360, reported 3D IoU ranges from to , while 2D mIoU ranges from to .
SplatFormer treats an optimized Gaussian scene as a structured point set and applies a point transformer to refine positions, scales, rotations, opacity, and spherical-harmonic appearance (Chen et al., 2024). The output preserves the same number of splats as the input. The method is designed for out-of-distribution camera viewpoints, especially high-elevation views not represented during initial low-elevation 3DGS optimization. Its residual architecture initially behaves as an identity mapping and is trained with image-space and LPIPS losses. On ShapeNet-OOD, SplatFormer reports $27.98$ PSNR, $0.920$ SSIM, and $0.136$ LPIPS, compared with $20.21$, $0.763$, and $0.242$ for standard 3DGS.
DSS, GaussianPU, PointGauss, and SplatFormer illustrate different roles for point splatting:
- Geometry optimization: point positions and normals are refined through differentiable rendering.
- Cardinality-controlled reconstruction: the number of output points is fixed before optimization.
- Semantic labeling: labels are attached to splats and projected consistently.
- Representation refinement: a learned point transformer corrects the 3D splat set without changing its cardinality.
Registration is another important application. ReFineR uses differentiable point splatting to align a single image with a pre-reconstructed point cloud by optimizing only the camera pose (Insalaco et al., 12 Nov 2026). It renders normals, depth, reflectivity, or grayscale channels and compares them with image gradients using GOM. Automatic point-cloud rescaling balances rotational and translational pose sensitivity through the eigenvalues of the reprojection Jacobian. The method is intended for refinement from a reasonable initial pose, not global localization.
6. Non-optical and cross-domain point splatting
The point-splatting paradigm extends beyond conventional camera images when the target domain has a meaningful spatial kernel and compositing rule.
RadSplatter adapts anisotropic Gaussian representations to radio-frequency radiomap extrapolation (Wang et al., 18 Feb 2025). Gaussian primitives represent virtual radio scatterers with position, covariance, complex attenuation, phase, and radio-oriented spherical-harmonic coefficients. A relaxed-mean scheme selects trainable centers from noisy point clouds, while camera-free MLPs map 3D Gaussian attributes to 2D radio beamspace. Contributions are complex-valued and are summed before RSS is computed:
This differs from optical splatting because the target is beamspace rather than pixels, and the primitive attributes represent attenuation, phase, and multipath rather than RGB radiance. On synthetic Shanghai and real Hangzhou data, RadSplatter reports MAE values of $7.564$ dB and $7.035$ dB, respectively.
3D Point Splatting for mmWave Radar, or 3DPS, derives an oriented-point renderer from the solid-angle radar equation (Armouti et al., 10 Sep 2026). Each point carries position, area, orientation, and a six-parameter ITU-R P.2040 material model. Its complex scattering contribution is splatted into range bins with a precomputed Hann-window PSF. A 15-tap range splat reduces work from for direct ADC synthesis to . The resulting complex range profiles can be transformed through standard FFT pipelines into ADC, complex range profiles, and range–azimuth outputs without retraining. Across six outdoor ColoRadar scenes, 3DPS reports mean held-out range–azimuth Pearson correlation of $0.587$ and approximately three minutes of training per scene on an RTX 4090.
4D Point Splatting extends this approach to moving humans and micro-Doppler (Armouti et al., 5 Nov 2026). Oriented radar points carry material, normal, and velocity; static and human points are splatted into range and Doppler bins; and the output is a complex radar tensor before downstream signal processing. Human velocity is derived from a 14-joint model with learned bounded temporal residuals. Closed-form Doppler kernels avoid rendering every chirp for every point. The method reports median correlations of $0.940$ and $0.830$ in range–azimuth for HuPR and RT-Pose, respectively, and micro-Doppler spectrogram correlations of $0.988$ and $0.966$.
These radar systems demonstrate that “splatting” need not imply image formation. The essential structure is the same: discrete spatial primitives, finite support in a target domain, differentiable weighted accumulation, and optimization through the resulting signal. The target domain may be pixels, feature planes, voxels, radio beams, range bins, Doppler bins, or combinations thereof.
7. Limitations, trade-offs, and research directions
Point splatting avoids explicit mesh connectivity and can preserve open, disconnected, noisy, or nonmanifold geometry. It also supports differentiable optimization, efficient GPU rasterization, topology changes, arbitrary-view rendering, and controlled explicit outputs. These advantages come with recurring limitations.
The quality of a splat representation depends on point density, initialization, footprint scale, visibility handling, and regularization. Sparse regions produce holes unless splats are enlarged, but enlarged splats can blur silhouettes and merge unrelated surfaces. Fixed kernels are especially vulnerable to nonuniform sampling, compression artifacts, and mixed-depth neighborhoods. Learned covariance, scale, opacity, or point displacement can mitigate these effects but introduces training dependence and possible hallucination.
Splatting does not inherently provide watertightness, mesh connectivity, or exact surface visibility. Disk representations can overlap or leave gaps; Gaussian-center clouds are not automatically metrically accurate surfaces; and image-optimized Gaussian positions may explain appearance without corresponding to a complete physical surface. The Waterloo urban reconstruction study explicitly distinguishes Gaussian image formation from metric geometry: extracted point clouds consist of Gaussian positional means and do not directly sample covariance ellipsoids or reconstruct a surface (Gao et al., 2024).
Differentiable visibility remains only piecewise smooth. Depth ordering, support boundaries, occlusion, and rasterization caches can change discretely. Radar systems introduce additional ambiguities involving phase periodicity, material non-identifiability, multipath, sensor-specific beam patterns, and magnitude-only supervision. Learned feature decoders can fill holes but may produce unreal content when geometric evidence is absent. Point transformers and 3D U-Nets improve context reasoning but increase memory, preprocessing cost, or inference latency.
Current research directions include:
- Adaptive support: learning anisotropic footprints, local bandwidths, and geometry-aware kernels.
- Efficient context aggregation: replacing heavy 3D networks with local averaging, coarse-grid aggregation, or sparse attention.
- Explicit uncertainty: representing confidence in geometry, visibility, appearance, or material parameters.
- Temporal consistency: extending static splats to dynamic 4D scenes without frame-to-frame jitter.
- Cross-modal rendering: jointly supporting RGB, depth, LiDAR, radio, radar, and semantic outputs.
- Phase-aware optimization: exploiting complex-valued supervision in RF and radar rather than fitting only magnitudes.
- Metric reconstruction: separating appearance-optimized splat centers from metrically reliable geometry.
- Large-scale deployment: addressing memory, tiling, hierarchical representations, and unbounded environments.
- Topology and surface integration: converting splat fields into meshes or other structured geometry without losing thin or open surfaces.
Point splatting consequently occupies an intermediate position between discrete point rendering, surfel-based surface representation, mesh rasterization, Gaussian radiance fields, neural feature reconstruction, and physics-based signal simulation. Its unifying principle is the replacement of zero-dimensional samples with finite, composable, and often differentiable spatial contributions.