RoMaP: Robust 3D Masked Part Editing

• RoMaP is a 3D editing framework that enables precise, text-guided, part-level modifications in Gaussian splatting scenes.
• It integrates a geometry-aware 3D mask generator (3D-GALP) with a regularized SDS objective and SLaMP-based 2D editing for localized changes.
• Empirical results demonstrate improved CLIP similarity and segmentation accuracy, highlighting its efficacy over previous diffusion-based methods.

Searching arXiv for “RoMaP” and closely related titles to ground the article in the cited papers. RoMaP, short for “Robust 3D-Masked Part-level Editing in 3D Gaussian Splatting with Regularized Score Distillation Sampling,” is a framework for precise, text-guided local editing of 3D Gaussian splatting scenes, with a particular emphasis on part-level modifications that are both drastic and spatially confined. It addresses two coupled difficulties in localized 3D editing: unreliable part-level mask construction from inconsistent multi-view 2D segmentations, and the ambiguity of Score Distillation Sampling (SDS) when the objective is to change only a narrowly specified region while preserving the rest of the scene. RoMaP combines a robust 3D mask generation module, 3D-Geometry Aware Label Prediction (3D-GALP), with a regularized SDS objective and a rectified-flow-based 2D part editor, SLaMP, to obtain multi-view-consistent, localized edits on reconstructed and generated Gaussian scenes (Kim et al., 15 Jul 2025).

1. Problem setting and design objectives

The task addressed by RoMaP is local 3D editing under natural-language instructions such as “left eye green, right eye blue” or “hair made of butterflies.” Given a 3D scene represented by 3D Gaussian splats $\Omega$ and a text instruction describing an edit on a specific part, the goal is to modify only the corresponding part while precisely localizing the edit in 3D, allowing large semantic changes, and preserving the appearance and geometry of all non-target regions across all views (Kim et al., 15 Jul 2025).

RoMaP is motivated by two failure modes of prior diffusion-based 3D editing pipelines. First, multi-view 2D segmentation is unreliable at part level. Existing methods typically render multiple views, run 2D segmentation per view, and lift masks back to 3D. The data block notes that this is often sufficient for whole objects but breaks for small or ambiguous parts because occlusions and appearance changes across views make per-view part segmentation inconsistent; hard per-Gaussian labels also ignore the mixed or soft ownership that arises near part boundaries. Second, SDS is implicit rather than explicitly local: it does not directly encode “change only this region,” so it can yield subtle, diffuse, or misplaced changes, especially for low-probability edits.

The central diagnosis behind RoMaP is therefore twofold: accurate part-level 3D editing requires a robust, geometry-aware 3D mask that respects view-dependent ambiguity and soft labels, and it requires a regularized SDS objective with a clear directional anchor for the desired edited appearance.

2. Framework structure and data flow

RoMaP operates on a 3D Gaussian scene $\Omega$, a textual edit instruction, and optionally textual segmentation prompts naming the target part. Its output is a modified Gaussian set $\Omega'$ in which only the target part is edited according to the text, while context is preserved elsewhere (Kim et al., 15 Jul 2025).

The framework is organized around two main modules.

Module	Function	Core mechanism
3D-GALP	Robust 3D mask generation	SH-parameterized, view-dependent label prediction with neighbor consistency
Regularized SDS	Local 3D editing	Masked SDS, SLaMP-based $\mathcal{L}_1$ anchor, and Gaussian prior removal

Conceptually, the pipeline proceeds as follows. A set of multi-view images is rendered from $\Omega$. For each view, a text-conditioned attention map for the target part is extracted via SD3 with dense attention post-processing and treated as a pseudo 2D segmentation. Per-Gaussian label coefficients are then fitted so that their 2D projections approximate these attention maps, and the label field is refined in 3D to obtain a robust mask $\mathcal{M}_{3D}$. That mask is projected to 2D to define view-wise masks $\mathcal{M}_{2D}$, which are used both to generate anchor images with SLaMP and to restrict optimization during editing. The editing stage then optimizes Gaussian parameters with a regularized SDS objective that combines masked SDS and a masked $\mathcal{L}_1$ anchor loss.

This decomposition separates localization from modification. A plausible implication is that RoMaP treats segmentation quality not as a preprocessing convenience but as a first-class optimization variable, because inaccurate masking is itself a primary source of edit leakage.

3. 3D-GALP and robust 3D mask generation

RoMaP extends the standard 3D Gaussian representation by attaching label coefficients to each Gaussian. A Gaussian splat is represented as

$$\Omega_i = \{ \mathbf{p}_i, \mathbf{s}_i, \mathbf{q}_i, \alpha_i, \mathbf{c}_i, \mathbf{r}_i \},$$

where $\mathbf{p}_i$ is the center, $\Omega$0 the scale, $\Omega$1 the rotation, $\Omega$2 the opacity, $\Omega$3 the SH coefficients for view-dependent color, and $\Omega$4 the newly introduced SH label coefficients (Kim et al., 15 Jul 2025).

The crucial modeling decision is that labels, like colors, are view dependent. For a view direction $\Omega$5,

$\Omega$6

and differentiable rasterization yields a rendered label image $\Omega$7. This allows a Gaussian near a part boundary to express view-dependent label variation rather than being forced into a single hard class.

Pseudo 2D supervision is obtained from diffusion-based attention maps. A rendered view $\Omega$8 is passed to Stable Diffusion 3 with rectified flow and dense attention extraction to obtain a token-aligned attention map $\Omega$9. The attention map is then normalized via softmax with a temperature and refined with CascadePSP. Alignment between rendered labels and attention-derived pseudo masks is imposed through an $\Omega'$0 term of the form

$\Omega'$1

RoMaP then refines labels through a label-softness-aware procedure. For each Gaussian, it computes a variance $\Omega'$2 of $\Omega'$3 over view directions and an entropy $\Omega'$4 from the distribution over discrete part labels. Label softness is defined as

$\Omega'$5

High $\Omega'$6 identifies view-dependent, label-ambiguous Gaussians, typically near boundaries; low $\Omega'$7 identifies stable interior Gaussians. Anchors are selected from both extremes: the top $\Omega'$8 highest-softness Gaussians and the bottom $\Omega'$9 lowest-softness Gaussians.

Neighbor consistency is enforced in 3D through the 3D-GALP loss: $\mathcal{L}_1$0 This spatial regularization smooths the SH label field while preserving transitions aligned with 3D part boundaries.

The significance of 3D-GALP lies in its rejection of a purely image-space view of part segmentation. The segmentation lives in 3D Gaussian space, explicitly models mixed ownership near boundaries, and uses 3D neighbor structure to resolve noisy view-wise pseudo labels.

4. Regularized SDS and SLaMP-based local editing

RoMaP’s editing stage is built around a regularized SDS objective tailored to local 3D Gaussian editing. The paper defines a loss that combines masked SDS with a masked $\mathcal{L}_1$1 anchor term between a rendered prior-removed image and a SLaMP-edited image. The conceptual division of labor is explicit in the data block: masked SDS provides semantic direction and text alignment, while the masked $\mathcal{L}_1$2 term provides an exact pixel-space target for the edited region (Kim et al., 15 Jul 2025).

A central component is SLaMP, “Scheduled Latent Mixing and Part editing,” a 2D diffusion-based editor based on SD3 with rectified flow. SLaMP is designed to modify only the target region, preserve the rest of the image nearly identically, and handle large prompt-driven changes robustly. Its latent update is

$\mathcal{L}_1$3

Inside the target region, the diffusion process edits freely according to the new prompt. Outside the target region, the latent is interpolated toward the original latent according to a time-dependent schedule $\mathcal{L}_1$4. The data block states that $\mathcal{L}_1$5 starts small and then sharply increases at a chosen timestep $\mathcal{L}_1$6, balancing strong local change with context preservation.

The corresponding anchor loss is

$\mathcal{L}_1$7

where $\mathcal{L}_1$8 is the rendered prior-removed image and $\mathcal{L}_1$9 is the SLaMP-edited target appearance.

RoMaP also introduces Gaussian prior removal. For Gaussians in the target region, the original color SH coefficients are replaced by neutral values, and opacity or other parameters may be relaxed. This weakens the tendency of SDS to remain close to the initial local appearance, making drastic edits more feasible. In addition, gradients for non-target Gaussians are masked or frozen. Together, prior removal, masked losses, and the SLaMP anchor define a local editing regime in which changes are both stronger and better confined.

5. Integration with 3D Gaussian Splatting and empirical results

RoMaP is integrated directly into standard 3D Gaussian Splatting. Rendering remains differentiable through the Gaussian parameters, and edits are applied by optimizing color SH coefficients, opacity, and potentially geometry-related parameters such as position and scale in the target region. Because optimization is performed directly in 3D, multi-view consistency is preserved at the representation level rather than imposed post hoc (Kim et al., 15 Jul 2025).

The reported experiments cover reconstructed Gaussian scenes from IN2N and NeRF-Art, generated Gaussian scenes from GaussianDreamer, GSGEN, and RFDS, and complex scenes from 3D-OVS and LERF for segmentation evaluation. Editing baselines include IN2N, ViCA-NeRF, PDS, GaussCtrl, GaussianEditor, and DGE. Evaluation uses CLIP similarity, CLIP$\Omega$0, BLIP-VQA, TIFA, and user studies assessing alignment, fidelity, and part-level accuracy.

The main quantitative findings reported in the data block are as follows. RoMaP achieves CLIP $\Omega$1, CLIP$\Omega$2 $\Omega$3, BLIP-VQA $\Omega$4, and TIFA $\Omega$5, compared with best baselines “around” $\Omega$6, $\Omega$7, $\Omega$8, and $\Omega$9, respectively. In user studies, RoMaP is preferred approximately $\mathcal{M}_{3D}$0 of the time for editing and approximately $\mathcal{M}_{3D}$1 for generation scenarios, whereas baselines receive approximately $\mathcal{M}_{3D}$2–$\mathcal{M}_{3D}$3. For segmentation on complex 3D-OVS scenes, 3D-GALP achieves mIoU $\mathcal{M}_{3D}$4, compared with $\mathcal{M}_{3D}$5 for LeGaussian and $\mathcal{M}_{3D}$6 for LangSplat.

The ablations isolate the effect of each component. Baseline SDS without 3D mask or anchor yields CLIP $\mathcal{M}_{3D}$7 and CLIP$\mathcal{M}_{3D}$8 $\mathcal{M}_{3D}$9. Adding the 3D-GALP mask raises these to $\mathcal{M}_{2D}$0 and $\mathcal{M}_{2D}$1. Using masked $\mathcal{M}_{2D}$2 without SDS gives $\mathcal{M}_{2D}$3 and $\mathcal{M}_{2D}$4. The full regularized SDS objective reaches $\mathcal{M}_{2D}$5 and $\mathcal{M}_{2D}$6. This indicates that robust 3D masks materially improve directional control and that the anchor loss is crucial for strong edits.

These results support the paper’s broader claim that robust part-level editing requires both reliable 3D localization and a non-ambiguous optimization target. A plausible implication is that RoMaP’s gains derive less from any single diffusion component than from the alignment between segmentation, anchor generation, and masked 3D optimization.

6. Limitations, scope, and acronym disambiguation

The paper identifies several limitations. RoMaP depends on diffusion models, especially SD3, both for attention-map quality and for SLaMP editing. If SD3 misinterprets a scene or is biased, segmentation and editing can fail. Extremely complex or self-intersecting geometries can break the reliability of 360° attention-based segmentation. Poor underlying reconstructions also degrade performance because blurry or sparse Gaussian scenes are harder for SD3 to parse and for 3D-GALP to segment accurately. The full pipeline is computationally expensive, involving multi-view rendering, diffusion-based attention extraction, SLaMP inversion, and iterative SDS optimization on a single A100 (Kim et al., 15 Jul 2025).

Within the literature represented in the supplied material, RoMaP should also be distinguished from similarly named but unrelated systems. “RoMA” in remote sensing denotes “Rotation-aware Multi-scale Autoregressive learning,” a self-supervised pretraining framework for Mamba-based remote sensing foundation models (Wang et al., 13 Mar 2025). “ROMA” in wireless communications denotes “ROtary and Movable Antenna,” a multi-antenna architecture with joint antenna movement and array rotation for multi-user MIMO (Zhang et al., 23 Jan 2025). The acronym similarity is superficial; RoMaP is a 3D Gaussian editing framework rather than a remote-sensing pretraining method or an antenna architecture.

In the context of 3D content creation, RoMaP’s significance lies in how it combines 3D Gaussian splatting, diffusion-based attention, and rectified-flow part editing into a unified local editing system. Its contribution is not merely the use of SDS for 3D optimization, but the coupling of geometry-aware mask construction, masked optimization, and explicit 2D anchors for localized semantic change.