MeshSplatting: Mesh-Gaussian 3D Splatting

• MeshSplatting is a framework that integrates mesh representations with Gaussian splatting to produce explicit, editable 3D surfaces for realistic rendering.
• It leverages differentiable rendering techniques and mesh extraction pipelines to jointly optimize geometry and appearance with precise mesh-aligned parameterizations.
• The approach enables real-time simulation and AR/VR integration, improving metrics like PSNR, SSIM, and LPIPS while reducing memory footprint.

MeshSplatting refers to a family of frameworks in computational graphics and vision that combine mesh-structured representations with primitive-based splatting methods—most notably, 3D Gaussian splatting—to yield explicit, editable, and differentiable mesh surfaces suitable for high-fidelity rendering, physical simulation, and interactive modeling. While Gaussian splatting was originally developed for rapid, high-quality novel-view synthesis from point clouds, mesh-based extensions provide geometric connectivity, enable mesh extraction and direct deformation, improve physical plausibility, and facilitate integration with mesh-centric pipelines in AR/VR and game engines. Key advancements encompass mesh-aligned parameterizations, normal supervision, topology-aware dynamic tracking, and differentiable mesh rasterization, leading to workflows where geometry and appearance are jointly optimized for photorealistic rendering and robust downstream applications.

1. Foundations of MeshSplatting: Splat-Mesh Parameterizations

MeshSplatting re-anchors the primitives of Gaussian splatting—notably, continuous 3D Gaussians or generalized exponentials—directly onto the vertices or faces of an explicit mesh. Each splat possesses a learned center $\mu_i \in \mathbb{R}^3$, covariance $\Sigma_i$, and appearance features such as color $c_i$, opacity $\alpha_i$, or spherical harmonic coefficients.

Mesh-parameterized splats typically employ barycentric coordinates $(\alpha, \beta, \gamma)$ enforcing $\alpha + \beta + \gamma = 1$, so that

$\mu_i = \alpha v_a + \beta v_b + \gamma v_c$

for triangle face $(v_a, v_b, v_c)$. The covariance aligns with the mesh geometry via local frames (e.g., $R_V$ normal matrix) and axis scales, often regularized to ensure surface flattening ($s_1 \ll s_2, s_3$) (Waczyńska et al., 2 Feb 2024, Gao et al., 7 Feb 2024).

Generalizations such as dynamic generalized exponential splatting (GES) enable non-Gaussian primitive shapes, further reducing the needed splat count for sharp representations (Zhao et al., 14 Nov 2024). Mesh-aligned representations permit direct propagation of mesh edits and deformations to the splatting parameters.

2. Differentiable Rendering and Mesh Extraction

MeshSplatting leverages differentiable rendering techniques for joint geometry-appearance optimization. Each pixel's color is produced by compositing projected primitives, typically accumulating along image rays and using analytic integration for Gaussian or exponential splats. In the mesh setting, projected ellipses or polygons are formed per splat using camera intrinsics/extrinsics, with screen-space splatting realized via alpha blending or ray tracing (Byrski et al., 15 Mar 2025).

To recover explicit meshes for downstream use, several pipelines are adopted:

• Mesh extraction from an implicit scalar field (e.g. a signed distance function $\varphi(x)$ estimated from splat densities) via Marching Cubes; normals are estimated as $\nabla\varphi(x)$ for high-quality per-vertex orientation (Krishnan et al., 14 Jan 2025).
• Splat-to-mesh conversion, forming mesh fans (triangle soups) from iso-contours of each splat’s covariance in its local frame (Tobiasz et al., 11 Feb 2025). Mesh faces are further optimized using photometric and SSIM losses.
• Topology-aware mesh construction, preserving connectivity via adjacency graphs inherited from the initial mesh, enabling stable vertex tracking in dynamic sequences (Guo et al., 1 Dec 2025).
• Restricted Delaunay triangulation, which enforces mesh connectivity for triangle soups initialized from sparse point clouds (Held et al., 7 Dec 2025).

3. Regularization, Deformation, and Topology Consistency

MeshSplatting benefits from regularization strategies engineered for explicit surface alignment:

• Normal supervision encourages per-splat normals to match the mesh face or SDF gradient, with corresponding loss terms (e.g., $L_{\mathrm{nc}} = \sum (1 - n_i \cdot n_{f(i)})$) ensuring orientation conformity (Choi et al., 11 Oct 2024).
• Scale regularization combats axis ballooning, ensuring splats remain locally surface-aligned; flattening terms force one covariance eigenvalue to zero (or near-zero).
• Projection losses (Choi et al., 11 Oct 2024) and SDF-alignment losses (Zhao et al., 14 Nov 2024) further constrain splat locations to the mesh surface or isosurface.

In dynamic scenarios, connectivity consistency across frames is maintained via temporal regularizers:

• Edge-length consistency ($L_{\mathrm{len}}$), rigidity constraints ($L_{\mathrm{rigid}}$), and quaternion-based rotation consistency ($L_{\mathrm{rot}}$) operate on mesh adjacency graphs, ensuring that mesh sequences remain topologically stable and suitable for animation and tracking (Guo et al., 1 Dec 2025).

Deformation mechanisms leverage mesh-driven edits—translations, rotations, scaling, bending, and stretching—propagating per-vertex modifications to attached splats through affine maps and local Jacobians, integrated with position-based dynamics or as-rigid-as-possible deformation solvers (B, 9 Jul 2025, Gao et al., 7 Feb 2024).

4. Integration with Ray Tracing, Mesh Pipelines, and Real-Time Engines

MeshSplatting resolves the incompatibilities of point-based splatting with mesh-centric graphics pipelines by producing explicit, connected, and opaque meshes:

• Ray tracing is unlocked via per-splat mesh conversion, enabling coherent handling of shadows, reflections, transmission, and all light interactions in standard engines such as Blender and Nvdiffrast (Byrski et al., 15 Mar 2025, Tobiasz et al., 11 Feb 2025).
• Depth-buffer rasterization and occlusion culling are fully supported, enabling high-performance rendering for AR/VR and game applications; meshes can be exported to Unity, Unreal, or other physic engines (Held et al., 7 Dec 2025).
• Editable mesh-based controls allow for interactive operations (dragging, sculpting, keypoint tracking, etc.) without retraining (Waczyńska et al., 2 Feb 2024, Gao et al., 7 Feb 2024).
• Real-time rates are consistently reported: MeshSplatting, MeshGS, and related frameworks achieve 60–220 FPS for scenes with up to hundreds of thousands of splats/triangles on modern GPUs (Gao et al., 7 Feb 2024, Held et al., 7 Dec 2025).

5. Quantitative Performance, Fidelity, and Comparative Analysis

MeshSplatting techniques are systematically benchmarked against point-based splatting, neural radiance field methods (NeRF), and prior mesh-based approaches:

• Mesh-based splatting regularly improves PSNR (by $+$0.69 to %%%%16$c_i$17%%%%), SSIM, and LPIPS on standard scene datasets (Mip-NeRF360, Tanks & Temples, Deep Blending) while reducing splat count and memory footprint by $25$–$37\%$ (Zhao et al., 14 Nov 2024, Choi et al., 11 Oct 2024, Held et al., 7 Dec 2025).
• Table: Representative Metrics

Method	PSNR ↑	SSIM ↑	LPIPS ↓	Mesh Editable?
3DGS (point)	28.69	0.870	0.220	No
MeshGS (Choi et al., 11 Oct 2024)	25.7	0.890	0.243	Yes
MeshSplats (2DGS init)	28.08	0.817	0.229	Yes
MeshSplatting (Held et al., 7 Dec 2025)	24.78	0.728	0.310	Yes (opaque)

Ablation studies show that geometric regularization and connectivity refinement are critical for surface fidelity and artifact suppression. Zero-shot transfer, temporal coherence, and surface detail (thin structures, hole-free continuity) are improved in mesh-oriented pipelines.

6. Applications, Limitations, and Extensions

MeshSplatting advances the suitability of neural scene representations for real-time graphics, simulation, and modeling. Principal applications include:

• Physics-based simulation, collision detection, and robotics (scene meshes are directly usable in physical engines).
• Animation, AR/VR, character rigging, and keypoint tracking (topology-aware frameworks enable stable per-frame mesh sequences and interactive control) (Guo et al., 1 Dec 2025).
• Interactive design and high-fidelity asset construction for film and games.
• Surface editing, segmentation, and object extraction via mesh manipulation.

Documented limitations include incomplete coverage in sparse or unobserved regions, the inability of opaque mesh splats to represent true translucency, and potential lack of watertightness or manifold guarantees (Held et al., 7 Dec 2025). Future research aims to incorporate neural textures, topological refinement, and hybrid mesh/volumetric representations; efficiency improvements continue via dynamic resolution, splat reduction, and surface-aligned regularization (Zhao et al., 14 Nov 2024).

7. Historical Context and Related Techniques

MeshSplatting builds on foundational works in differentiable surface rendering (Rasterize-then-Splat (Cole et al., 2021)), mesh-based adaptation of splatting primitives (Waczyńska et al., 2 Feb 2024), and the integration of point-based methods with mesh connectivity and dynamic control. Recent efforts focus on generalizable pipelines for sparse-view reconstruction (Chang et al., 25 Aug 2025), mesh-to-splat-to-mesh round-tripping for editing, and dynamic topology preservation for 4D sequence modeling (Guo et al., 1 Dec 2025).

Collectively, these developments interlink fast view synthesis, explicit geometric modeling, and differentiable optimization, forging a tractable route to neural scene representations compatible with industry-standard 3D workflows, large-scale physical simulation, and interactive design.