--- title: 'DriveSplat: Geometry-Aware Dynamic Scene Splatting' url: https://www.emergentmind.com/topics/drivesplat type: topic --- # DriveSplat: Geometry-Aware Dynamic Scene Splatting DriveSplat is a reconstruction method for dynamic driving scenes that combines neural Gaussian representations with dynamic-static decoupling, geometry-enhanced partitioned optimization for the static background, deformable neural Gaussians for non-rigid actors, and monocular depth and normal priors [2508.15376]. It was introduced to address a specific failure mode of earlier dynamic 3D Gaussian Splatting pipelines: although they separate static and dynamic content, they still optimize the background largely by view fitting, which limits robustness to novel viewpoints and weakens geometric fidelity. In DriveSplat, the central design choice is to preserve explicit Gaussian rendering while strengthening cross-view geometry through octree anchors, near/middle/far partitioning aligned with driving trajectories, and auxiliary geometric supervision [2508.15376]. ## 1. Problem formulation and design goals DriveSplat targets street-scale reconstruction in settings with fast-moving actors, large static backgrounds spanning near-to-far ranges, and frequent motion blur [2508.15376]. These conditions jointly stress appearance fidelity and geometry accuracy: distant structures often lack dense LiDAR support, moving objects violate static-scene assumptions, and conventional 3DGS optimization can accumulate redundant Gaussians that fit training views but generalize poorly to unseen viewpoints. The method is organized around four stated goals: decoupled static/dynamic modeling under a unified neural Gaussian representation; geometry-enhanced, region-wise partitioned optimization for the static background; deformable neural Gaussians for non-rigid dynamic actors; and depth and normal priors from pretrained monocular models to enforce cross-view geometric consistency [2508.15376]. Its inputs are RGB frames, camera calibration from either structure-from-motion or the dataset, an initialized fused point cloud from SfM, LiDAR, or DUSt3R, per-actor tracked bounding boxes and poses, and monocular priors from DepthAnything-V2, ZoeDepth, and Omnidata [2508.15376]. The method assumes that driving viewpoints predominantly translate along a principal axis with limited rotation, which is standard in ego-vehicle trajectories [2508.15376]. This assumption is not incidental: it drives the region-wise partitioning strategy that distinguishes near, middle, and far background regions. A common misconception is that dynamic-static decoupling alone is sufficient for driving-scene splatting. DriveSplat is explicitly motivated by the opposite position: prior decoupling strategies still rely on standard 3DGS background fitting and therefore do not adequately strengthen geometry relationships across views [2508.15376]. ## 2. Neural Gaussian representation and differentiable rendering DriveSplat represents the scene with neural Gaussians whose per-primitive attributes include mean $\boldsymbol{\mu}\in\mathbb{R}^3$, covariance $\boldsymbol{\Sigma}\in\mathbb{R}^{3\times 3}$, opacity $\alpha\in[0,1]$, and color $\mathbf{c}\in\mathbb{R}^3$ [2508.15376]. The covariance is parameterized through rotation and scale as $$ \boldsymbol{\Sigma}=\mathbf{R}\mathbf{S}^2\mathbf{R}^\top. $$ For the static background, occupied voxel centers serve as octree anchors $\boldsymbol{\mu}_a$, and each level $l$ has a learnable offset $\mathcal{X}_l$ and scaling factor $f_a$: $$ \{\boldsymbol{\mu}_0,\ldots,\boldsymbol{\mu}_{O-1}\} = \boldsymbol{\mu}_a+\{\mathcal{X}_0,\ldots,\mathcal{X}_{O-1}\}\cdot f_a. $$ This creates a multi-LOD Gaussian scaffold tied to a fused multi-sensor point cloud rather than to unconstrained Gaussian proliferation [2508.15376]. Projection follows the standard camera model. A world-space point transforms as $$ \mathbf{x}_c=\mathbf{R}_c(\mathbf{x}_w-\mathbf{t}_c), $$ and projects with intrinsics $\mathbf{K}$ to $$ \mathbf{u}=\Pi(\mathbf{x}_c)= \begin{bmatrix} f_x x_c/z_c + c_x\\ f_y y_c/z_c + c_y \end{bmatrix}. $$ Using the Jacobian $\mathbf{J}$ of the projection at the Gaussian center, the screen-space covariance is $$ \boldsymbol{\Sigma}_{2D}=\mathbf{J}\,\boldsymbol{\Sigma}\,\mathbf{J}^\top. $$ The resulting 2D elliptical footprint is $$ g_i(\mathbf{u})=\exp\!\left(-\tfrac{1}{2}(\mathbf{u}-\mathbf{u}_i)^\top \boldsymbol{\Sigma}_{2D,i}^{-1} (\mathbf{u}-\mathbf{u}_i)\right). $$ DriveSplat composites depth-sorted Gaussians front-to-back with weights $$ w_i=\alpha_i\,g_i(\mathbf{u})\prod_{j\phi_2>1$ [2508.15376]. Near and middle regions therefore receive finer voxels, while far voxels remain coarser to avoid redundancy. The significance of this partitioned background optimization is twofold. First, it ties reconstruction density to the projective structure of ego-vehicle motion rather than to a scene-agnostic grid. Second, it turns the fused point cloud into a geometric prior rather than merely a seed set. The paper explicitly frames this as going beyond per-view fitting by Gaussian addition: octree anchors tied to multi-sensor structure, region-wise adaptive voxels, and depth and normal priors collectively act as cross-view geometric constraints [2508.15376]. DriveSplat further supervises the background with monocular priors. Relative depth $\mathbf{D}_r$ comes from DepthAnything-V2, metric depth $\mathbf{D}_m$ from ZoeDepth, and normals $\mathbf{N}_m$ from Omnidata [2508.15376]. The rendering loss is $$ \mathcal{L}_r=\|\mathbf{I}-\mathbf{I}_{gt}\|_1+\lambda\,\mathcal{L}_{SSIM}(\mathbf{I},\mathbf{I}_{gt}), $$ the depth loss is either the scale-invariant correlation loss $$ \mathcal{L}_{depth}= \bigg\| \frac{\operatorname{Cov}(D,D_r)} {\sqrt{\operatorname{Var}(D)\operatorname{Var}(D_r)}} \bigg\|_1, $$ or an $L_1$ term to metric depth $\|D-D_m\|_1$, and the normal loss is $$ \mathcal{L}_{normal}=\|N-N_m\|_1+\big(1-\langle \hat{N},\hat{N}_m\rangle\big). $$ A dynamic mask loss $\mathcal{L}_{mask}$ is added, giving the full objective $$ \mathcal{L}= \mathcal{L}_r+ \lambda_d\,\mathcal{L}_{depth}+ \lambda_n\,\mathcal{L}_{normal}+ \lambda_m\,\mathcal{L}_{mask}. $$ These losses formalize the paper’s claim that geometry should be supervised directly rather than left to emerge from photometric fitting alone [2508.15376]. ## 4. Dynamic actor decoupling and deformable neural Gaussians DriveSplat separates dynamic actors from the static background using tracked actor bounding boxes and poses [2508.15376]. With LiDAR, each frame’s point cloud is transformed to an actor-local coordinate system by $\mathbf{T}_{obj}^{-1}$, points inside the actor’s axis-aligned bounding box define the actor cloud $\mathbf{P}_{obj}$, and the residual points form the static background $\mathbf{P}_s$ [2508.15376]. When LiDAR is absent, the method uses COLMAP for static points and random seeding within actor boxes for dynamic points [2508.15376]. Rigid dynamic actors are modeled in local coordinates and transformed to world space using tracked poses. If a local Gaussian has position $\boldsymbol{\mu}_o$ and rotation $\mathbf{R}_o$, and the tracked global pose at time $t$ is $(\mathbf{R}_t,\mathbf{T}_t)$, then $$ \boldsymbol{\mu}_t=\mathbf{R}_t\,\boldsymbol{\mu}_o+\mathbf{T}_t,\qquad \mathbf{R}_t^{world}=\mathbf{R}_o\,\mathbf{R}_t^\top. $$ This provides explicit motion control for rigid vehicles and similar actors [2508.15376]. Non-rigid actors, such as pedestrians and cyclists, are handled with deformable neural Gaussians. DriveSplat gathers anchor and offset points across octree levels, $$ \mathcal{P}=\big\{\boldsymbol{\mu}_i+\mathcal{X}_j \mid i=1,\dots,N,\; j=1,\dots,O\big\}, $$ and feeds them with time/context cues into a deformation network $\mathcal{F}_\theta$. The network predicts $$ \boldsymbol{\mu}_t=\boldsymbol{\mu}+\Delta\boldsymbol{\mu}_t,\qquad \boldsymbol{\Sigma}_t=\mathbf{A}_t\,\boldsymbol{\Sigma}\,\mathbf{A}_t^\top,\qquad \mathbf{R}_t^{def}=\Delta\mathbf{R}_t\,\mathbf{R}, $$ where $(\Delta\boldsymbol{\mu}_t,\mathbf{A}_t,\Delta\mathbf{R}_t)=\mathcal{F}_\theta(t,\text{feat}(\mathcal{P}))$ [2508.15376]. This time-conditioned deformation field is meant to capture non-rigid motion while preserving actor identity and coherence. The decoupling strategy is therefore asymmetric. The static background is heavily geometry-regularized and partition-aware, whereas the dynamic branch emphasizes tracked rigid transformations plus local deformation where necessary. This design distinguishes DriveSplat from static-scene octree splatting and from dynamic pipelines that treat all motion with a uniform time-varying primitive model. ## 5. Optimization protocol, ablations, and reported performance DriveSplat is optimized per scene for 30,000 iterations with Adam in PyTorch on a single NVIDIA L20 GPU [2508.15376]. For novel-view synthesis evaluation, every fourth frame is held out [2508.15376]. The datasets reported are Waymo Open, with eight sequences across varying weather and traffic and two OmniRe sequences for non-rigid ablations, and KITTI, with three challenging sequences [2508.15376]. The main quantitative results reported for image quality are summarized below [2508.15376]. | Dataset / protocol | Reconstruction | Novel-view synthesis | |---|---|---| | Waymo | 36.08 PSNR, 0.943 SSIM, 0.079 LPIPS | 34.41 PSNR, 0.928 SSIM, 0.087 LPIPS | | KITTI | 28.59 PSNR, 0.895 SSIM, 0.100 LPIPS | 24.53 PSNR, 0.767 SSIM, 0.142 LPIPS | On Waymo novel-view synthesis, these numbers exceed AD-GS at 33.46 PSNR, Octree-GS at 32.13 PSNR, and StreetGS at 30.87 PSNR [2508.15376]. The paper also reports cleaner vehicles, sharper static backgrounds, and robust behavior under challenging viewpoint shifts such as ego offset [2508.15376]. The ablations are central to the method’s interpretation. For initialization, SfM+LiDAR gives the best novel-view synthesis at 34.41 PSNR, 0.928 SSIM, and 0.087 LPIPS; LiDAR alone gives the best depth but lacks tall and distant coverage; DUSt3R is densest but suffers from misaligned scale and position [2508.15376]. For partitioned optimization, DriveSplat with BPO reaches 34.41/0.928/0.087, versus 33.82/0.926/0.093 without BPO [2508.15376]. For depth supervision, relative depth $\mathbf{D}_r$ gives PSNR 34.41 with AbsRel 0.172, while metric depth $\mathbf{D}_m$ gives the best depth with AbsRel 0.121 but slightly worse PSNR at 33.23 [2508.15376]. Normal priors reduce normal MAE and RMSE and increase cosine similarity to 0.504, while also improving PSNR [2508.15376]. The dynamic module is particularly important on pedestrian-heavy sequences. The full model reports 37.93 PSNR, 0.971 SSIM, and 0.029 LPIPS, surpassing OmniRe at 37.26, 0.971, and 0.032 [2508.15376]. This directly supports the claim that deformable neural Gaussians matter beyond simple rigid actor transforms. The main limitations are also explicit. DriveSplat requires actor tracking and bounding boxes; performance depends on the quality of depth and normal priors and on principal-axis estimation; metric depth can trade off rendering quality; and extreme non-linear camera motion may weaken the near/mid/far partitioning assumption [2508.15376]. These limitations are structural rather than incidental: each corresponds to one of the priors that makes the system work. ## 6. Place in the driving-scene splatting literature DriveSplat belongs to a broader line of Gaussian-splatting methods for autonomous driving, but it occupies a distinct position within that line. AutoSplat introduced geometric constraints on road and sky, reflected Gaussian consistency for vehicle symmetry, and residual spherical harmonics for dynamic appearance, with an emphasis on realistic scene reconstruction and lane-change robustness [2407.02598]. SplatFlow moved toward self-supervised dynamic Gaussian splatting using Neural Motion Flow Fields and removed the need for tracked 3D bounding boxes, while targeting RGB, depth, and flow synthesis in dynamic scenes [2411.15482]. ReconDrive and UniSplat shifted the field toward feed-forward 4DGS generation using, respectively, VGGT-based hybrid heads and a unified 3D latent scaffold with spatio-temporal fusion [2603.07552][2511.04595]. SpectralSplat kept a feed-forward backbone but disentangled appearance from geometry, enabling controllable relighting and appearance transfer [2604.03462]. IDSplat made instance decomposition explicit, using zero-shot language-grounded video tracking lifted to 3D with lidar and optimized rigid SE(3) object trajectories without human annotations [2511.19235]. Against this backdrop, DriveSplat is most precisely characterized as a geometry-centered, optimization-based system whose distinctive contribution is not feed-forward inference, language control, or appearance disentanglement, but the combination of dynamic-static decoupling with geometry-enhanced background optimization [2508.15376]. This suggests that its enduring significance lies less in being a terminal architecture than in establishing a strong reference point for geometry-aware driving-scene splatting: later methods repeatedly address issues that DriveSplat left open, such as annotation-free dynamic decomposition, feed-forward scalability, and appearance control, while preserving the underlying premise that explicit Gaussian representations remain effective for large-scale autonomous-driving reconstruction [2411.15482][2603.07552][2604.03462]. A further misconception is to treat DriveSplat as merely a dynamic extension of standard 3DGS. The paper’s own framing is narrower and more technical: it is a decoupled neural Gaussian system for driving scenes whose main novelty is the strengthening of geometry relations in the background and the explicit handling of non-rigid actors through a learnable deformation network [2508.15376]. In that sense, DriveSplat is best read as a method for enforcing geometric discipline in a domain where photometric fitting alone is insufficient. Source: https://www.emergentmind.com/topics/drivesplat