DriveSplat: Geometry-Aware Dynamic Scene Splatting

• DriveSplat is a neural Gaussian reconstruction method that decouples static and dynamic elements for accurate driving scene modeling.
• It employs geometry-enhanced, region-wise partitioned optimization with octree anchors and monocular depth and normal priors to improve metrics like PSNR and SSIM.
• The method integrates deformable neural Gaussians for non-rigid actors, ensuring robust reconstructions in scenarios with motion blur and fast-moving objects.

DriveSplat is a reconstruction method for dynamic driving scenes that combines neural Gaussian representations with dynamic-static decoupling, geometry-enhanced partitioned optimization for the static background, deformable neural Gaussians for non-rigid actors, and monocular depth and normal priors (Wang et al., 21 Aug 2025). It was introduced to address a specific failure mode of earlier dynamic 3D Gaussian Splatting pipelines: although they separate static and dynamic content, they still optimize the background largely by view fitting, which limits robustness to novel viewpoints and weakens geometric fidelity. In DriveSplat, the central design choice is to preserve explicit Gaussian rendering while strengthening cross-view geometry through octree anchors, near/middle/far partitioning aligned with driving trajectories, and auxiliary geometric supervision (Wang et al., 21 Aug 2025).

1. Problem formulation and design goals

DriveSplat targets street-scale reconstruction in settings with fast-moving actors, large static backgrounds spanning near-to-far ranges, and frequent motion blur (Wang et al., 21 Aug 2025). These conditions jointly stress appearance fidelity and geometry accuracy: distant structures often lack dense LiDAR support, moving objects violate static-scene assumptions, and conventional 3DGS optimization can accumulate redundant Gaussians that fit training views but generalize poorly to unseen viewpoints.

The method is organized around four stated goals: decoupled static/dynamic modeling under a unified neural Gaussian representation; geometry-enhanced, region-wise partitioned optimization for the static background; deformable neural Gaussians for non-rigid dynamic actors; and depth and normal priors from pretrained monocular models to enforce cross-view geometric consistency (Wang et al., 21 Aug 2025). Its inputs are RGB frames, camera calibration from either structure-from-motion or the dataset, an initialized fused point cloud from SfM, LiDAR, or DUSt3R, per-actor tracked bounding boxes and poses, and monocular priors from DepthAnything-V2, ZoeDepth, and Omnidata (Wang et al., 21 Aug 2025).

The method assumes that driving viewpoints predominantly translate along a principal axis with limited rotation, which is standard in ego-vehicle trajectories (Wang et al., 21 Aug 2025). This assumption is not incidental: it drives the region-wise partitioning strategy that distinguishes near, middle, and far background regions. A common misconception is that dynamic-static decoupling alone is sufficient for driving-scene splatting. DriveSplat is explicitly motivated by the opposite position: prior decoupling strategies still rely on standard 3DGS background fitting and therefore do not adequately strengthen geometry relationships across views (Wang et al., 21 Aug 2025).

2. Neural Gaussian representation and differentiable rendering

DriveSplat represents the scene with neural Gaussians whose per-primitive attributes include mean $\boldsymbol{\mu}\in\mathbb{R}^3$, covariance $\boldsymbol{\Sigma}\in\mathbb{R}^{3\times 3}$, opacity $\alpha\in[0,1]$, and color $\mathbf{c}\in\mathbb{R}^3$ (Wang et al., 21 Aug 2025). The covariance is parameterized through rotation and scale as

$$\boldsymbol{\Sigma}=\mathbf{R}\mathbf{S}^2\mathbf{R}^\top.$$

For the static background, occupied voxel centers serve as octree anchors $\boldsymbol{\mu}_a$, and each level $l$ has a learnable offset $\mathcal{X}_l$ and scaling factor $f_a$:

$$\{\boldsymbol{\mu}_0,\ldots,\boldsymbol{\mu}_{O-1}\} = \boldsymbol{\mu}_a+\{\mathcal{X}_0,\ldots,\mathcal{X}_{O-1}\}\cdot f_a.$$

This creates a multi-LOD Gaussian scaffold tied to a fused multi-sensor point cloud rather than to unconstrained Gaussian proliferation (Wang et al., 21 Aug 2025).

Projection follows the standard camera model. A world-space point transforms as

$\boldsymbol{\Sigma}\in\mathbb{R}^{3\times 3}$0

and projects with intrinsics $\boldsymbol{\Sigma}\in\mathbb{R}^{3\times 3}$1 to

$\boldsymbol{\Sigma}\in\mathbb{R}^{3\times 3}$2

Using the Jacobian $\boldsymbol{\Sigma}\in\mathbb{R}^{3\times 3}$3 of the projection at the Gaussian center, the screen-space covariance is

$\boldsymbol{\Sigma}\in\mathbb{R}^{3\times 3}$4

The resulting 2D elliptical footprint is

$\boldsymbol{\Sigma}\in\mathbb{R}^{3\times 3}$5

DriveSplat composites depth-sorted Gaussians front-to-back with weights

$\boldsymbol{\Sigma}\in\mathbb{R}^{3\times 3}$6

yielding

$\boldsymbol{\Sigma}\in\mathbb{R}^{3\times 3}$7

Accumulated alpha is

$\boldsymbol{\Sigma}\in\mathbb{R}^{3\times 3}$8

Surface normals can then be estimated by differentiating depth or via oriented Gaussian frames (Wang et al., 21 Aug 2025).

This representation is technically notable because the geometry pathway and renderer remain standard enough to preserve the efficiency of 3DGS, while the optimization space is structured by anchors, LODs, and priors. The paper’s emphasis is therefore not on altering splat rasterization itself, but on altering how Gaussians are initialized and constrained.

3. Geometry-enhanced partitioned optimization of the static background

The background module is organized around what the paper describes as geometry-enhanced, region-wise partitioned optimization (Wang et al., 21 Aug 2025). The motivation is that a single global voxel size is poorly matched to driving data: image-space contributions are dominated by near and middle distances, and a uniform grid either wastes capacity in the far field or blurs close-range structure.

DriveSplat first estimates the principal direction of the static point cloud with PCA, projects points onto that axis, then fits a 1D Gaussian mixture model and derives thresholds $\boldsymbol{\Sigma}\in\mathbb{R}^{3\times 3}$9 and $\alpha\in[0,1]$0 that partition anchors into near, middle, and far regions (Wang et al., 21 Aug 2025). Let $\alpha\in[0,1]$1 denote the base voxel size at octree level $\alpha\in[0,1]$2, with $\alpha\in[0,1]$3 and $\alpha\in[0,1]$4 derived from normalized inverse point density. The region-wise voxel size is

$\alpha\in[0,1]$5

with predefined scalars $\alpha\in[0,1]$6 (Wang et al., 21 Aug 2025). Near and middle regions therefore receive finer voxels, while far voxels remain coarser to avoid redundancy.

The significance of this partitioned background optimization is twofold. First, it ties reconstruction density to the projective structure of ego-vehicle motion rather than to a scene-agnostic grid. Second, it turns the fused point cloud into a geometric prior rather than merely a seed set. The paper explicitly frames this as going beyond per-view fitting by Gaussian addition: octree anchors tied to multi-sensor structure, region-wise adaptive voxels, and depth and normal priors collectively act as cross-view geometric constraints (Wang et al., 21 Aug 2025).

DriveSplat further supervises the background with monocular priors. Relative depth $\alpha\in[0,1]$7 comes from DepthAnything-V2, metric depth $\alpha\in[0,1]$8 from ZoeDepth, and normals $\alpha\in[0,1]$9 from Omnidata (Wang et al., 21 Aug 2025). The rendering loss is

$\mathbf{c}\in\mathbb{R}^3$0

the depth loss is either the scale-invariant correlation loss

$\mathbf{c}\in\mathbb{R}^3$1

or an $\mathbf{c}\in\mathbb{R}^3$2 term to metric depth $\mathbf{c}\in\mathbb{R}^3$3, and the normal loss is

$\mathbf{c}\in\mathbb{R}^3$4

A dynamic mask loss $\mathbf{c}\in\mathbb{R}^3$5 is added, giving the full objective

$\mathbf{c}\in\mathbb{R}^3$6

These losses formalize the paper’s claim that geometry should be supervised directly rather than left to emerge from photometric fitting alone (Wang et al., 21 Aug 2025).

4. Dynamic actor decoupling and deformable neural Gaussians

DriveSplat separates dynamic actors from the static background using tracked actor bounding boxes and poses (Wang et al., 21 Aug 2025). With LiDAR, each frame’s point cloud is transformed to an actor-local coordinate system by $\mathbf{c}\in\mathbb{R}^3$7, points inside the actor’s axis-aligned bounding box define the actor cloud $\mathbf{c}\in\mathbb{R}^3$8, and the residual points form the static background $\mathbf{c}\in\mathbb{R}^3$9 (Wang et al., 21 Aug 2025). When LiDAR is absent, the method uses COLMAP for static points and random seeding within actor boxes for dynamic points (Wang et al., 21 Aug 2025).

Rigid dynamic actors are modeled in local coordinates and transformed to world space using tracked poses. If a local Gaussian has position $$\boldsymbol{\Sigma}=\mathbf{R}\mathbf{S}^2\mathbf{R}^\top.$$0 and rotation $$\boldsymbol{\Sigma}=\mathbf{R}\mathbf{S}^2\mathbf{R}^\top.$$1, and the tracked global pose at time $$\boldsymbol{\Sigma}=\mathbf{R}\mathbf{S}^2\mathbf{R}^\top.$$2 is $$\boldsymbol{\Sigma}=\mathbf{R}\mathbf{S}^2\mathbf{R}^\top.$$3, then

$$\boldsymbol{\Sigma}=\mathbf{R}\mathbf{S}^2\mathbf{R}^\top.$$4

This provides explicit motion control for rigid vehicles and similar actors (Wang et al., 21 Aug 2025).

Non-rigid actors, such as pedestrians and cyclists, are handled with deformable neural Gaussians. DriveSplat gathers anchor and offset points across octree levels,

$$\boldsymbol{\Sigma}=\mathbf{R}\mathbf{S}^2\mathbf{R}^\top.$$5

and feeds them with time/context cues into a deformation network $$\boldsymbol{\Sigma}=\mathbf{R}\mathbf{S}^2\mathbf{R}^\top.$$6. The network predicts

$$\boldsymbol{\Sigma}=\mathbf{R}\mathbf{S}^2\mathbf{R}^\top.$$7

where $$\boldsymbol{\Sigma}=\mathbf{R}\mathbf{S}^2\mathbf{R}^\top.$$8 (Wang et al., 21 Aug 2025). This time-conditioned deformation field is meant to capture non-rigid motion while preserving actor identity and coherence.

The decoupling strategy is therefore asymmetric. The static background is heavily geometry-regularized and partition-aware, whereas the dynamic branch emphasizes tracked rigid transformations plus local deformation where necessary. This design distinguishes DriveSplat from static-scene octree splatting and from dynamic pipelines that treat all motion with a uniform time-varying primitive model.

5. Optimization protocol, ablations, and reported performance

DriveSplat is optimized per scene for 30,000 iterations with Adam in PyTorch on a single NVIDIA L20 GPU (Wang et al., 21 Aug 2025). For novel-view synthesis evaluation, every fourth frame is held out (Wang et al., 21 Aug 2025). The datasets reported are Waymo Open, with eight sequences across varying weather and traffic and two OmniRe sequences for non-rigid ablations, and KITTI, with three challenging sequences (Wang et al., 21 Aug 2025).

The main quantitative results reported for image quality are summarized below (Wang et al., 21 Aug 2025).

Dataset / protocol	Reconstruction	Novel-view synthesis
Waymo	36.08 PSNR, 0.943 SSIM, 0.079 LPIPS	34.41 PSNR, 0.928 SSIM, 0.087 LPIPS
KITTI	28.59 PSNR, 0.895 SSIM, 0.100 LPIPS	24.53 PSNR, 0.767 SSIM, 0.142 LPIPS

On Waymo novel-view synthesis, these numbers exceed AD-GS at 33.46 PSNR, Octree-GS at 32.13 PSNR, and StreetGS at 30.87 PSNR (Wang et al., 21 Aug 2025). The paper also reports cleaner vehicles, sharper static backgrounds, and robust behavior under challenging viewpoint shifts such as ego offset (Wang et al., 21 Aug 2025).

The ablations are central to the method’s interpretation. For initialization, SfM+LiDAR gives the best novel-view synthesis at 34.41 PSNR, 0.928 SSIM, and 0.087 LPIPS; LiDAR alone gives the best depth but lacks tall and distant coverage; DUSt3R is densest but suffers from misaligned scale and position (Wang et al., 21 Aug 2025). For partitioned optimization, DriveSplat with BPO reaches 34.41/0.928/0.087, versus 33.82/0.926/0.093 without BPO (Wang et al., 21 Aug 2025). For depth supervision, relative depth $$\boldsymbol{\Sigma}=\mathbf{R}\mathbf{S}^2\mathbf{R}^\top.$$9 gives PSNR 34.41 with AbsRel 0.172, while metric depth $\boldsymbol{\mu}_a$0 gives the best depth with AbsRel 0.121 but slightly worse PSNR at 33.23 (Wang et al., 21 Aug 2025). Normal priors reduce normal MAE and RMSE and increase cosine similarity to 0.504, while also improving PSNR (Wang et al., 21 Aug 2025).

The dynamic module is particularly important on pedestrian-heavy sequences. The full model reports 37.93 PSNR, 0.971 SSIM, and 0.029 LPIPS, surpassing OmniRe at 37.26, 0.971, and 0.032 (Wang et al., 21 Aug 2025). This directly supports the claim that deformable neural Gaussians matter beyond simple rigid actor transforms.

The main limitations are also explicit. DriveSplat requires actor tracking and bounding boxes; performance depends on the quality of depth and normal priors and on principal-axis estimation; metric depth can trade off rendering quality; and extreme non-linear camera motion may weaken the near/mid/far partitioning assumption (Wang et al., 21 Aug 2025). These limitations are structural rather than incidental: each corresponds to one of the priors that makes the system work.

6. Place in the driving-scene splatting literature

DriveSplat belongs to a broader line of Gaussian-splatting methods for autonomous driving, but it occupies a distinct position within that line. AutoSplat introduced geometric constraints on road and sky, reflected Gaussian consistency for vehicle symmetry, and residual spherical harmonics for dynamic appearance, with an emphasis on realistic scene reconstruction and lane-change robustness (Khan et al., 2024). SplatFlow moved toward self-supervised dynamic Gaussian splatting using Neural Motion Flow Fields and removed the need for tracked 3D bounding boxes, while targeting RGB, depth, and flow synthesis in dynamic scenes (Sun et al., 2024). ReconDrive and UniSplat shifted the field toward feed-forward 4DGS generation using, respectively, VGGT-based hybrid heads and a unified 3D latent scaffold with spatio-temporal fusion (Yu et al., 8 Mar 2026, Shi et al., 6 Nov 2025). SpectralSplat kept a feed-forward backbone but disentangled appearance from geometry, enabling controllable relighting and appearance transfer (Herau et al., 3 Apr 2026). IDSplat made instance decomposition explicit, using zero-shot language-grounded video tracking lifted to 3D with lidar and optimized rigid SE(3) object trajectories without human annotations (Lindström et al., 24 Nov 2025).

Against this backdrop, DriveSplat is most precisely characterized as a geometry-centered, optimization-based system whose distinctive contribution is not feed-forward inference, language control, or appearance disentanglement, but the combination of dynamic-static decoupling with geometry-enhanced background optimization (Wang et al., 21 Aug 2025). This suggests that its enduring significance lies less in being a terminal architecture than in establishing a strong reference point for geometry-aware driving-scene splatting: later methods repeatedly address issues that DriveSplat left open, such as annotation-free dynamic decomposition, feed-forward scalability, and appearance control, while preserving the underlying premise that explicit Gaussian representations remain effective for large-scale autonomous-driving reconstruction (Sun et al., 2024, Yu et al., 8 Mar 2026, Herau et al., 3 Apr 2026).

A further misconception is to treat DriveSplat as merely a dynamic extension of standard 3DGS. The paper’s own framing is narrower and more technical: it is a decoupled neural Gaussian system for driving scenes whose main novelty is the strengthening of geometry relations in the background and the explicit handling of non-rigid actors through a learnable deformation network (Wang et al., 21 Aug 2025). In that sense, DriveSplat is best read as a method for enforcing geometric discipline in a domain where photometric fitting alone is insufficient.