SplatFill: 3D Inpainting for 3D Gaussian Splatting

• SplatFill is a 3D scene inpainting framework that combines a high-quality 2D inpainted reference with depth priors and object-aware supervision for editing 3D Gaussian scenes.
• It integrates monocular depth estimates and Selective Guided Inpainting (SGI) to correct geometric discrepancies and ensure multi-view consistency during novel-view rendering.
• Empirical results demonstrate improved full-image quality metrics and a 24.5% reduction in training time compared to prior NeRF and 3D Gaussian Splatting methods.

SplatFill is a 3D scene inpainting method for 3D Gaussian Splatting (3DGS) that targets object removal, occlusion completion, and geometry completion required for consistent novel-view rendering. It takes multi-view RGB images, camera poses, and binary masks indicating regions that should be removed and then plausibly filled, and it produces a complete editable 3DGS scene representation rather than a collection of independently edited 2D images (Dahaghin et al., 9 Sep 2025). Its central design combines a single high-quality 2D inpainted reference view, monocular depth priors, object-aware supervision, and an iterative consistency-aware refinement procedure called Selective Guided Inpainting, or SGI, to improve geometric placement, visual coherence, and efficiency in masked regions (Dahaghin et al., 9 Sep 2025).

1. Problem setting and motivation

SplatFill addresses 3D scene inpainting in a multi-view 3DGS setting where missing regions arise from occlusion or scene editing. Typical examples include object removal from a reconstructed scene, filling content revealed after editing, recovering surfaces that were previously occluded, and completing geometry needed for consistent novel-view rendering (Dahaghin et al., 9 Sep 2025). The output must remain plausible under viewpoint change, which makes the task materially different from standard 2D image inpainting.

The method is motivated by three recurring failure modes in prior pipelines. First, independent 2D inpainting across views often yields slight cross-view discrepancies that become ghosting, geometry conflicts, blurry splats, and instability after 3D optimization. Second, plausible 2D completions do not by themselves determine correct 3D placement, which can lead to floating Gaussians, incorrect depths, broken boundaries, and inconsistent geometry. Third, prior NeRF-based and 3DGS-based inpainting systems often produce oversmoothed textures, blurred details, and artifacts, especially for viewpoints far from a single reference image (Dahaghin et al., 9 Sep 2025).

Within the paper’s taxonomy of prior work, NeRF-based methods such as SPIn-NeRF, OR-NeRF, MVIP-NeRF, Inpaint3D, NeRFiller, and reference-guided NeRF inpainting address parts of the problem but incur trade-offs in speed, detail, or multi-view consistency. Prior 3DGS methods are described as falling into two common paradigms: single-reference conditioning, which can fail under large viewpoint changes, and multi-view aggregation of inpainted images, which can blur or artifact when the 2D generations are not fully consistent. SplatFill is formulated as a response to those limitations by combining sharp-detail initialization from one inpainted view with multi-view geometric guidance, object-aware supervision, and selective iterative refinement rather than global re-inpainting (Dahaghin et al., 9 Sep 2025).

2. Pipeline and system components

The pipeline is explicitly organized into seven stages. It first selects a reference view and performs 2D inpainting of its masked region. It then estimates monocular depth maps for the inpainted reference and the other masked views, groups pixels by depth level to obtain reliable depth priors, applies instance segmentation on training images, initializes and optimizes Gaussians in missing regions using the inpainted reference image together with depth and object-aware constraints, detects inconsistent views by comparing monocular depth and rendered depth, and then repeats refinement until the views satisfy geometric and visual consistency requirements (Dahaghin et al., 9 Sep 2025).

Several external modules are part of the system definition. The reference-view completion is produced by a diffusion-based 2D inpainting method from Esser et al. (2024). Monocular depth is obtained with Depth Anything, and instance segmentation masks are obtained using SAM2. The 3DGS optimizer is implemented on a modified MCMC framework whose rasterizer is extended to render depth (Dahaghin et al., 9 Sep 2025).

The method is not a new renderer from scratch. It builds on standard 3DGS and retains explicit Gaussian scene representation, differentiable splat rasterization, alpha compositing, and optimization of geometry and appearance parameters. Its novelty lies in inpainting-specific supervision and refinement. The initial missing content is introduced through one selected reference image, whose masked area is filled in 2D and then used as the first direct supervision signal for the hidden region. The paper is explicit that it does not provide a separate closed-form formula for Gaussian initialization in the masked region; instead, new or updated Gaussians in missing areas are placed and refined through optimization under photometric, depth, and object-aware signals (Dahaghin et al., 9 Sep 2025).

An initial baseline reconstruction is established with 8000 steps before selective refinement begins. This division between baseline optimization and later targeted correction is central to the method’s efficiency claim, because only inconsistent views and subregions are revisited during SGI (Dahaghin et al., 9 Sep 2025).

3. Depth-guided and object-aware supervision

Depth supervision is one of the two principal supervision channels. The paper argues that a 2D inpainted image provides appearance but not necessarily correct 3D placement, so monocular depth priors are used to place inpainted content in plausible depth ranges, reduce floating or outlier Gaussians, improve volumetric consistency across views, and identify inconsistent regions during refinement (Dahaghin et al., 9 Sep 2025).

The core depth term is the Soft Depth Clustering Loss (SDCL). Instead of direct metric depth supervision, which would require scale-and-shift alignment between monocular depth and rendered depth, SDCL partitions the estimated depth map into evenly spaced depth bins, constructs a mask $\mathcal{M}_k$ for each bin, computes the mean rendered depth $\mu_k$ inside that bin, and penalizes deviations of rendered depths from the within-bin mean. This avoids explicit metric alignment and weights distant bins less heavily because monocular depth is less reliable there (Dahaghin et al., 9 Sep 2025). The paper presents this as a key novelty and argues that it preserves finer details for far objects better than conventional metric depth supervision.

A second depth term, the Crop-Focused Depth Loss (CFDL), concentrates supervision near the inpainted area. Every 9 iterations, a crop is taken around the inpainted region, randomly expanded to include context, processed with the same binning strategy as SDCL, and weighted by a Gaussian-like function that emphasizes pixels near the center of the inpainted area. The total depth objective is the sum of SDCL and CFDL with $\kappa = 25$ (Dahaghin et al., 9 Sep 2025).

The second supervision channel is object-aware. Each Gaussian is augmented with a 16-dimensional learnable feature vector, and rendered feature maps are grouped by SAM2 instance masks. The resulting Object-Aware Contrastive Loss (OACL) encourages features from the same segment to cluster and features from different segments to separate (Dahaghin et al., 9 Sep 2025). In the paper’s framing, depth indicates where content should be, while object priors indicate what belongs together structurally. This dual supervision is intended not only to improve reconstruction quality but also to support later tasks such as 3D object selection (Dahaghin et al., 9 Sep 2025).

A common misconception is that SplatFill relies only on depth. The paper’s design is explicitly bimodal: depth regularizes spatial placement, while object supervision regularizes structural coherence and object boundaries. The authors also note a limitation here: segmentation groups Gaussians by object, but the method does not yet segment and complete each 3D object independently (Dahaghin et al., 9 Sep 2025).

4. Consistency-aware refinement with Selective Guided Inpainting

The refinement stage, Selective Guided Inpainting (SGI), addresses the failure case in which a model optimized from a single inpainted reference remains inconsistent in views that are far from that reference. Rather than globally re-inpainting all masked content, SGI renders all training views from the current 3DGS model, compares rendered depth with monocular depth inside masked regions, and identifies the view with the largest cumulative inconsistency for targeted correction (Dahaghin et al., 9 Sep 2025).

The refinement signal is computed from the per-pixel absolute depth discrepancy

$E(x,y) = \left| D_{\text{rend}(x,y) - D_{\text{mono}(x,y) \right|.$

For each view, the error is summed over masked-region pixels, and the view with the largest cumulative error is selected as $v^*$. Within that view, the gradient magnitude

$G(x,y) = \left\| \nabla E(x,y) \right\|_2$

is used to localize sharp inconsistency boundaries such as misplaced surfaces and depth discontinuities (Dahaghin et al., 9 Sep 2025). The implementation uses the Sobel operator, erodes the base mask to reduce boundary artifacts, thresholds the gradient map, and applies morphological dilation to obtain a refined binary mask $B(x,y)$ that marks only the problematic subregion.

Only that selected subregion is re-inpainted in 2D. The updated image is then added to the reference set, and 3DGS optimization resumes. This process repeats until the discrepancies become small or geometric and visual consistency is reached (Dahaghin et al., 9 Sep 2025). The paper’s rationale is that global re-inpainting is both expensive and destabilizing: it may overwrite regions that are already correct and can inject inconsistent 2D generations into the 3D model. SGI preserves already coherent regions and concentrates computation on the largest residual errors.

This refinement policy is also one of the method’s primary efficiency mechanisms. The abstract attributes the training-time gain to a consistency-aware refinement scheme that selectively identifies and corrects inconsistent regions without disrupting the rest of the scene, and the reported reduction relative to GScream is 24.5% (Dahaghin et al., 9 Sep 2025).

5. Mathematical formulation and representation

SplatFill retains the standard 3DGS alpha-compositing renderer. For a pixel $p$, the rendered color is

$$C(p) = \sum_{i=1}^{|\mathcal{G}_p|} \mathbf{c}_i\, \alpha_i' \prod_{j=1}^{i-1} \left(1 - \alpha_j'\right),$$

where $\mathcal{G}_p$ is the set of Gaussians affecting $\mu_k$0, $\mu_k$1 is the color of Gaussian $\mu_k$2, and $\mu_k$3 is the effective opacity contribution of that Gaussian at the pixel (Dahaghin et al., 9 Sep 2025). Each Gaussian includes 3D position $\mu_k$4, covariance $\mu_k$5, color $\mu_k$6, opacity $\mu_k$7, spherical harmonics coefficients, and the additional 16-dimensional feature vector used for OACL (Dahaghin et al., 9 Sep 2025).

The per-bin SDCL term is

$\mu_k$8

and the weighted global SDCL is

$\mu_k$9

The full depth term is

$\kappa = 25$0

with $\kappa = 25$1 (Dahaghin et al., 9 Sep 2025).

The object-aware feature loss is

$\kappa = 25$2

with temperature

$\kappa = 25$3

where $\kappa = 25$4 (Dahaghin et al., 9 Sep 2025). All features are $\kappa = 25$5-normalized before loss computation.

The paper repeatedly refers to optimization of photometric parameters and supervision by the images, but it does not print a single full total-loss formula combining photometric, depth, and OACL terms in the provided text (Dahaghin et al., 9 Sep 2025). That omission matters for reproduction, because the paper’s novel losses are specified precisely while the exact global weighting of all objectives is not.

6. Evaluation, comparative performance, and empirical findings

The main benchmark is SPIn-NeRF, which contains 10 scenes and 100 multi-view images per scene with human-annotated object masks. For each scene, 60 training images contain the unwanted object and 40 test images show the scene without the object (Dahaghin et al., 9 Sep 2025). Evaluation uses PSNR, SSIM, LPIPS, and FID, computed both over the full image and over the masked region only (Dahaghin et al., 9 Sep 2025).

On full-image evaluation, SplatFill reports PSNR 20.46, SSIM 0.63, LPIPS 0.25, FID 29.76, and 34 min training time. The corresponding numbers reported for GScream are 20.45, 0.58, 0.28, 36.72, and 45 min; SPIn-NeRF reports 20.18, 0.46, 0.47, 58.78, and approximately 112.5 min; OR-NeRF reports 20.32, 0.54, 0.35, 38.69, and approximately 225 min (Dahaghin et al., 9 Sep 2025). The abstract summarizes these results as state-of-the-art perceptual quality and improved efficiency, with the training-time reduction relative to GScream quantified as 24.5% (Dahaghin et al., 9 Sep 2025).

On masked-region evaluation, SplatFill reports PSNR 15.67, SSIM 0.21, and LPIPS 0.54, essentially matching GScream on those three metrics while remaining stronger on the full-image evaluation (Dahaghin et al., 9 Sep 2025). The paper interprets this as evidence that SplatFill preserves or improves overall scene coherence without sacrificing masked-region competitiveness.

The ablations isolate the method’s main components. For depth-based supervision, the masked-region numbers are 15.14 / 0.18 / 0.56 without depth supervision, 15.22 / 0.19 / 0.56 with the GScream depth loss, and 15.67 / 0.21 / 0.54 with SplatFill’s full depth loss, indicating that SDCL + CFDL improves Gaussian placement and perceptual quality (Dahaghin et al., 9 Sep 2025). For the inpainting strategy, Single Reference gives 15.35 / 0.19 / 0.54, while Selective Guided Inpainting gives 15.67 / 0.21 / 0.54, supporting the claim that one reference is insufficient and that selective multi-view correction improves robustness (Dahaghin et al., 9 Sep 2025).

Qualitatively, the paper attributes to SplatFill sharper textures, better geometric coherence, fewer artifacts, and stronger consistency in difficult viewpoints. The examples highlighted include sharper edges of a stone bench, finer grass details behind fences, and more detailed grass around a manhole, especially where GScream becomes blurred (Dahaghin et al., 9 Sep 2025).

7. Position within related research, applications, and limitations

Within the broader splat-based and diffusion-based literature, SplatFill occupies a specific niche: explicit 3D scene inpainting for 3DGS. This distinguishes it from methods that emphasize other forms of control or lifting. ControlFill is a latent-diffusion 2D inpainting method with learned creation and removal conditions and a spatial guidance tensor, but it is explicitly described as not a 3D-aware, multi-view consistent, geometry-aware, or Gaussian-splatting method (Jeon, 6 Mar 2025). SplatDiff is a pixel-splatting-guided latent video diffusion model for novel view synthesis that fills disocclusions and reduces texture hallucination, but it is not formulated as explicit 3DGS scene inpainting (Zhang et al., 18 Feb 2025). FluSplat addresses sparse-view text-guided 3D editing without test-time optimization and emphasizes cross-view consistent editing rather than explicit filling of unseen or occluded regions (Huang et al., 21 Apr 2026). FSFSplatter improves sparse-view, pose-free Gaussian-splatting reconstruction through dense initialization, self-splitting densification, and geometry-enhanced optimization, but it is described as adjacent rather than a direct scene-completion method (Zhao et al., 3 Oct 2025). Splat Feature Solver assigns and denoises semantic features on existing splats without adding geometry (Xiong et al., 17 Aug 2025), and Splat-LOAM performs LiDAR odometry and mapping with local densification over observed surfaces rather than generative scene completion (Giacomini et al., 21 Mar 2025). This positioning suggests that SplatFill is one of the more explicit treatments of the 3DGS inpainting problem itself.

The method’s practical use cases include object removal, occlusion completion, scene editing, and content insertion. The supplementary material is described as showing that the same pipeline can be used for object insertion, using a text-prompted 2D inpainting result as a reference and propagating it into a coherent 3D scene (Dahaghin et al., 9 Sep 2025). The paper also lists AR/VR, game design, film post-production, and robotics among application domains that benefit from fast editable 3D scenes (Dahaghin et al., 9 Sep 2025).

The paper is explicit about several limitations. The initial reference view is selected randomly, and its quality strongly affects reconstruction quality. Object supervision is underused in the sense that the method does not yet segment and complete each 3D object independently. Illumination changes, including shadows and reflections, are not explicitly handled and can still produce inconsistencies across views (Dahaghin et al., 9 Sep 2025). These caveats delimit the scope of the reported gains: SplatFill is strongest when the reference inpainting is high quality, monocular depth is sufficiently informative, and the remaining inconsistencies are localized enough for SGI to correct selectively.

In summary, SplatFill is best characterized as a depth-guided 3DGS inpainting framework that begins from a single high-quality 2D inpainted reference, augments standard Gaussian optimization with SDCL, CFDL, and OACL, and then uses SGI to correct only those views and subregions where rendered geometry still disagrees with monocular depth. Its reported contribution is not a new renderer, but an inpainting-specific supervisory and refinement regime for producing geometrically coherent, sharper, and more efficient 3D scene completions in the 3DGS setting (Dahaghin et al., 9 Sep 2025).