InterGSEdit: Interactive 3D Gaussian Splatting

• The paper introduces a user-interactive pipeline that anchors diffusion-based 3D editing to a key view, ensuring semantic consistency across non-rigid edits.
• It employs a CLIP-driven semantic selection and constructs a 3D Geometry-Consistent Attention Prior to unproject 2D cross-attention into a shared 3D representation.
• An Attention Fusion Network dynamically balances 3D guidance with 2D detail recovery, mitigating artifacts while preserving geometric fidelity.

Searching arXiv for the primary InterGSEdit paper and closely related 3DGS editing work to ground the article in current literature. arxiv_search(query="InterGSEdit Interactive 3D Gaussian Splatting Editing with 3D Geometry-Consistent Attention Prior", max_results=5) InterGSEdit is a user-interactive 3D Gaussian Splatting editing framework for diffusion-based 3D editing that addresses two coupled failure modes of prior 3DGS editors: multi-view inconsistency and poor user controllability. Its central premise is that text-only editing is often ambiguous, especially for non-rigid semantic edits such as facial expressions, so independently edited views may realize the same prompt differently and then inject inconsistent semantics into the reconstructed 3DGS. InterGSEdit replaces that purely text-driven regime with a user-anchored pipeline in which a selected key edited view serves as the semantic anchor, semantically consistent reference views are chosen around it, a 3D Geometry-Consistent Attention Prior $GAP^{3D}$ is built from those views, and an Attention Fusion Network dynamically balances 3D consistency against 2D detail recovery during diffusion (Wen et al., 7 Jul 2025).

1. Problem formulation and scope

The method is positioned against a common limitation of recent 3D Gaussian Splatting editing systems: they can work reasonably well for rigid edits such as style transfer, but they often fail on non-rigid semantic edits. The paper attributes this to prompt ambiguity. When each rendered view is edited independently by a diffusion model, different views can converge to different local semantic realizations of the same instruction, and the resulting 3D update can produce blurred textures, local artifacts, semantic variations across viewpoints, or distorted geometry (Wen et al., 7 Jul 2025).

InterGSEdit also targets the control problem created by text-only editing. The paper describes such editing as a “one-shot deal,” because multiple plausible realizations may satisfy a prompt while leaving no direct mechanism for specifying which realization is intended. InterGSEdit addresses this by letting the user choose a key edited view that best matches the desired outcome, then propagating that preference consistently across the remaining views. This reframes 3DGS editing from independent prompt-conditioned image edits into a constrained multi-view propagation problem grounded in shared 3D geometry.

A plausible implication is that the framework is most relevant where semantic ambiguity is large and viewpoint consistency is critical, rather than in narrowly constrained appearance-only edits. The paper’s qualitative examples emphasize both rigid tasks and difficult non-rigid facial-expression edits.

2. End-to-end interactive editing pipeline

The overall pipeline has two stages. First, the original 3DGS scene $\mathcal{G}_{src}$ is rendered into multiple views $I_{src}=\mathcal{R}(\mathcal{G},v)$, and each view is initially edited by a diffusion model to obtain $I'_{edit}$. One of these edited images is then chosen by the user as the key view. Second, the system uses that key view as a semantic anchor to select semantically consistent reference views, constructs a 3D attention prior from them, and uses this prior to guide the final multi-view diffusion editing (Wen et al., 7 Jul 2025).

The framework is explicitly organized into three components: CLIP-based Semantic Consistency Selection (CSCS), construction of the 3D Geometry-Consistent Attention Prior $GAP^{3D}$, and an Attention Fusion Network (AFN). CSCS provides the user-interactive control mechanism; $GAP^{3D}$ converts selected 2D cross-attention evidence into a shared 3D prior over Gaussian primitives; AFN fuses the projected 3D attention with the diffusion model’s native 2D cross-attention during denoising.

This decomposition is significant because it separates semantic preference acquisition, geometry-consistent propagation, and detail-preserving synthesis. The paper’s core claim is not merely that interaction is added to 3DGS editing, but that user interaction is operationalized through a geometry-aware attention pipeline rather than through post hoc view selection alone.

3. CLIP-based Semantic Consistency Selection

CSCS is the mechanism that converts the user-selected key view into a reference set for subsequent editing. Instead of using a manually fixed subset of views or averaging all edited views indiscriminately, InterGSEdit computes semantic consistency in a CLIP embedding space. For the key view, the method computes an edited-image shift and a text-editing direction; their cosine similarity defines the key-view alignment score $s_{\text{key}}$. For any candidate view $v$, the same score $s_v$ is computed, and the deviation from the key view is

$\Delta s_v = |s_v - s_{\text{key}}|.$

Views are ranked by this deviation, and the top-$\mathcal{G}_{src}$0 smallest-deviation views form the reference set $\mathcal{G}_{src}$1 (Wen et al., 7 Jul 2025).

The selection is softened rather than thresholded. Each candidate view is assigned a weight

$\mathcal{G}_{src}$2

where $\mathcal{G}_{src}$3 controls the temperature. Semantically closer views therefore contribute more strongly, while slightly less similar views are not discarded outright. According to the paper, this suppresses inconsistent editing directions that normally arise across views in non-rigid tasks.

The role of CSCS is both algorithmic and interactive. Algorithmically, it constructs a reference pool biased toward semantic agreement with the key view. Interactively, it turns the user’s chosen view into the semantic specification of the edit. This suggests that controllability is not implemented as manual low-level editing of all views, but as high-level selection of the preferred semantic realization followed by automatic propagation.

4. 3D Geometry-Consistent Attention Prior and Attention Fusion Network

From the weighted reference views, InterGSEdit constructs the 3D attention prior $\mathcal{G}_{src}$4 by unprojecting 2D cross-attention maps into the 3D Gaussian representation. For Gaussian point $\mathcal{G}_{src}$5, the prior aggregates attention contributions over the selected views using the semantic weights $\mathcal{G}_{src}$6, together with the opacity contribution $\mathcal{G}_{src}$7 and transmittance $\mathcal{G}_{src}$8 induced by the 3DGS renderer. The paper characterizes this as a weighted Gaussian Splatting unprojection: semantically closer views contribute more strongly, so the 3D prior is both geometrically consistent and aligned with the user’s preferred edit (Wen et al., 7 Jul 2025).

Once formed on Gaussian points, the prior is projected back into each target view as a 3D-constrained attention map,

$\mathcal{G}_{src}$9

Every view therefore receives attention derived from the same shared 3D structure. In the paper’s terminology, this makes the attention “3D-constrained.”

InterGSEdit does not simply replace the diffusion model’s native 2D cross-attention with this projected 3D attention. The paper explicitly states that using only 3D-constrained attention can hurt fine details and reduce editing fidelity because the 3D prior may be too conservative or oversmoothed. To address this, AFN fuses $I_{src}=\mathcal{R}(\mathcal{G},v)$0 and $I_{src}=\mathcal{R}(\mathcal{G},v)$1 at each diffusion layer $I_{src}=\mathcal{R}(\mathcal{G},v)$2 through a learnable gating mechanism with a temporal bias

$I_{src}=\mathcal{R}(\mathcal{G},v)$3

This produces an adaptive policy: early in diffusion, when global structure and geometry are being established, the network is biased toward $I_{src}=\mathcal{R}(\mathcal{G},v)$4; later, as denoising progresses, the bias decays and the model relies more on $I_{src}=\mathcal{R}(\mathcal{G},v)$5 for fine-grained details and prompt-specific appearance.

The AFN is further trained with a KL regularization term that encourages alignment between the 2D and 3D attention distributions. The paper states that a larger $I_{src}=\mathcal{R}(\mathcal{G},v)$6 is used early so the attention is pulled toward the geometry-consistent prior, and that $I_{src}=\mathcal{R}(\mathcal{G},v)$7 is gradually reduced later to allow more detail-rich 2D guidance. A common misconception would be to view the method as a pure 3D-attention replacement scheme; the paper instead presents it as a staged fusion strategy whose effectiveness depends on preserving both geometry consistency and image-level edit richness.

5. 3DGS formalism, implementation, and optimization

For the underlying 3DGS representation, each Gaussian $I_{src}=\mathcal{R}(\mathcal{G},v)$8 has mean $I_{src}=\mathcal{R}(\mathcal{G},v)$9, covariance $I'_{edit}$0, color $I'_{edit}$1, and opacity $I'_{edit}$2. The covariance is decomposed as $I'_{edit}$3, and the Gaussian density is written as

$I'_{edit}$4

The rendering equation given in the paper is

$I'_{edit}$5

These definitions matter because the attention unprojection uses the same rasterization and transmittance structure of 3DGS to map 2D attention back onto Gaussian primitives (Wen et al., 7 Jul 2025).

Implementation-wise, the authors build InterGSEdit on ThreeStudio, use InfEdit as the diffusion editing backbone, and use the latent consistency model from HuggingFace. Experiments are conducted on scenes from the IN2N dataset. For local editing, the method leverages the segmentation pipeline and semantic tracking from GSEditor. Each editing task uses 20 random views, and 3DGS optimization runs for 800 to 1200 iterations depending on scene complexity. Experiments are performed on an NVIDIA RTX 4090 GPU.

The paper also states that AFN and the scene 3DGS are optimized jointly during denoising. The main hyperparameters mentioned are the selection size $I'_{edit}$6, the CLIP weighting temperature $I'_{edit}$7, the AFN bias coefficient $I'_{edit}$8, and the loss weights $I'_{edit}$9 and $GAP^{3D}$0. This indicates that both semantic reference selection and the 2D/3D attention trade-off are explicit optimization controls rather than incidental implementation details.

6. Evaluation, ablations, and significance

Evaluation uses three metrics: CLIP Similarity, CTIDS, and CDC. CLIP Similarity measures image-text alignment between the edited image embedding and the target prompt embedding. CTIDS compares the text-edit direction $GAP^{3D}$1 and image-edit direction $GAP^{3D}$2. CDC measures directional consistency across multi-view edits, following IN2N. The reported results show that InterGSEdit, with both ip2p and InfEdit backbones, outperforms baselines including IGS2GS, GSEditor, DGE, and VcEdit with CCM. The paper reports, for InterGSEdit with InfEdit, $GAP^{3D}$3 CLIP Similarity, $GAP^{3D}$4 CTIDS, and $GAP^{3D}$5 CDC, all described as the best among the compared methods (Wen et al., 7 Jul 2025).

The ablations clarify the role of each component. Without CSCS, the 3D prior is built from all views by averaging, and the paper reports obvious tooth artifacts and inconsistency in non-rigid facial edits. Without AFN, directly replacing 2D attention by 3D-constrained attention causes some regions to be under-edited and reduces fidelity to the prompt. An “Only 2D Attention” setting produces stronger editing but also more artifacts and less geometric consistency. The full system achieves the best balance, with the highest CDC and strong CTIDS while also improving CLIP Similarity.

Qualitatively, the paper shows advantages on both non-rigid and rigid tasks. In facial-expression edits such as “make him smile,” “make him angry,” and “make him laugh,” prior methods often produce inconsistent teeth, distorted mouths, or blurry facial details because different views converge to different expression states. InterGSEdit keeps the expression style consistent with the chosen key view and yields fewer artifacts. In appearance edits such as hair color or hairstyle changes, the method better preserves geometry while capturing the target semantics more faithfully.

Taken together, these results support the paper’s broader significance claim: InterGSEdit reframes 3DGS editing from purely text-driven, view-independent editing into a user-anchored, geometry-consistent attention propagation problem. Its distinct contribution is the interaction between CSCS and AFN: CSCS determines which multi-view evidence is semantically compatible with the user’s intent, while AFN enforces a staged compromise between 3D consistency and 2D detail recovery. This suggests that the framework’s main advance lies less in any single module than in the explicit coupling of user selection, semantic filtering, and geometry-aware attention fusion.