CoreEditor: Consistent 3D Scene Editing
- CoreEditor is a 3D text-driven editing framework that maintains cross-view consistency using correspondence-constrained attention and semantic guidance.
- It combines geometric projections with semantic similarity to accurately map pixels across multiple views.
- Its selective editing pipeline enables user-driven reference propagation to preserve style and prevent blurry artifacts.
to=python code to=python code to=python code code='import urllib.request, json, re, ssl, sys\nurl="http://export.arxiv.org/api/query?search_query=all:CoreEditor&start=0&max_results=3"\ntry:\n print(urllib.request.urlopen(url, timeout=10).read()[:500].decode())\nexcept Exception as e:\n print("ERR", e)\n' to=python code code: import urllib.request, json, re, ssl, sys url="http://export.arxiv.org/api/query?search_query=all:CoreEditor&start=0&max_results=3" try: print(urllib.request.urlopen(url, timeout=10).read()[:500].decode()) except Exception as e: print("ERR", e)
to=python code import urllib.request, json, re, ssl, sys url="http://export.arxiv.org/api/query?search_query=all:CoreEditor&start=0&max_results=3" try: print(urllib.request.urlopen(url, timeout=10).read()[:500].decode()) except Exception as e: print("ERR", e)
to=shell code python - <<'PY' import urllib.request url='http://export.arxiv.org/api/query?search_query=all:CoreEditor&start=0&max_results=3' try: data=urllib.request.urlopen(url, timeout=10).read()[:500] print(data.decode()) except Exception as e: print('ERR', e) PY
CoreEditor is a framework for text-driven 3D editing that modifies 3D scenes according to textual descriptions while maintaining cross-view consistency across multi-view inputs. It is introduced in "CoreEditor: Consistent 3D Editing via Correspondence-constrained Diffusion" (Zhu et al., 15 Aug 2025). The framework targets a central limitation of many existing text-to-3D editing systems: they adapt pre-trained 2D image editors to multi-view inputs, but without explicit control over multi-view information exchange they often fail to maintain cross-view consistency, which leads to insufficient edits and blurry details. CoreEditor addresses this with a correspondence-constrained attention mechanism, geometry- and semantic-supported correspondence modeling, and a selective editing pipeline in which a preferred single-view edit is propagated consistently across views.
1. Problem setting and overall design
Text-driven 3D editing is formulated as a multi-view editing problem in which a 3D scene is represented through multiple rendered views and then modified by a diffusion-based image editor. In this setting, visual fidelity depends not only on whether an edit matches the text prompt in each individual view, but also on whether the edited content remains geometrically and semantically coherent when the scene is rendered from novel viewpoints. CoreEditor is designed precisely around that requirement.
The framework has three main components. First, a Correspondence-Constrained Attention (CCA) module enforces direct interaction only between pixels that are expected to correspond across views. Second, a geometry and semantic co-supported correspondence construction process combines depth-based projection with feature-space similarity, rather than relying solely on geometric alignment. Third, a selective editing pipeline allows a user to choose a preferred result from multiple single-view candidates and then uses that result as a global reference during joint editing (Zhu et al., 15 Aug 2025).
A common misconception in multi-view diffusion editing is that geometric alignment alone is sufficient for consistency. CoreEditor explicitly rejects that assumption. Its formulation treats occlusion, sparse geometry, and out-of-frame content as failure cases for purely geometric matching, and therefore supplements geometry with semantic similarity estimated during denoising. This suggests that the framework treats 3D consistency as a joint geometric-semantic constraint rather than as a projection-only constraint.
2. Correspondence-constrained attention
The central mechanism in CoreEditor is CCA, described as an augmentation of the traditional self-attention mechanism in diffusion U-Nets. For a source pixel in source view , CCA constructs attention only over corresponding positions drawn from other views, rather than allowing unrestricted cross-view mixing. The attended feature is given as
$\begin{split} Q &= Z_s(x_s, y_s) \ K = V &= \{ Z_i(\mathcal{C}[i]) \mid i=1,\ldots,N \} \ P' &= \mathrm{softmax}\left(\frac{QK^\top}{\sqrt{d}} + M' \right)V . \end{split}$
Here, denotes the corresponding position in view , and is an attention mask that removes invalid or occluded correspondences. The key distinction from vanilla self-attention is explicit in the formulation: only specifically matched tokens, rather than the whole view, participate in cross-view attention for a given pixel (Zhu et al., 15 Aug 2025).
This design has two immediate consequences. First, it restricts denoising interactions to locations that are intended to remain mutually consistent, which is a stronger constraint than generic feature sharing. Second, it makes consistency a property enforced during the diffusion process itself, rather than a post hoc regularization objective. In the paper’s framing, this is the mechanism responsible for sharper details and improved cross-view agreement.
3. Correspondence construction from geometry and semantics
CoreEditor constructs correspondences in two stages. The geometric stage projects a source pixel into another view using depth and calibrated cameras:
Here, is the depth at in view , 0 is the intrinsic matrix, and 1 are the extrinsics for source and target views. A re-projection error is then used to identify unreliable matches, such as those induced by occlusion, and such matches are masked out.
When geometric correspondence is unavailable or unreliable, CoreEditor switches to semantic correspondence. It uses the diffusion model’s last-layer feature maps 2 and searches for the pixel in another view that maximizes cosine similarity:
3
Only correspondences with similarity greater than the threshold 4 are accepted, and the reported optimal value is 5 (Zhu et al., 15 Aug 2025).
The resulting correspondence set 6 therefore combines geometric and semantic matches, each accompanied by a validity mask. The paper’s ablations state that using only geometry or only semantic correspondences performs worse than combining both. This is significant because it reframes consistency not as a single-source matching problem, but as a co-supported estimation problem in which geometry provides precision when visible and semantics recovers alignment when visibility or projection fails.
4. Selective editing pipeline and reference attention
CoreEditor’s selective editing pipeline begins by inverting all source view images 7 into the diffusion model’s latent space and then performing vanilla text-to-image editing independently on those views. From the resulting candidates, the user selects a preferred edited image 8. The paper also notes that this selection can be automated by a preference predictor such as ImageReward. The chosen reference is then injected into joint consistent editing through Reference Attention (RA).
The RA formulation is
9
Here, $\begin{split} Q &= Z_s(x_s, y_s) \ K = V &= \{ Z_i(\mathcal{C}[i]) \mid i=1,\ldots,N \} \ P' &= \mathrm{softmax}\left(\frac{QK^\top}{\sqrt{d}} + M' \right)V . \end{split}$0 is the selected reference feature, $\begin{split} Q &= Z_s(x_s, y_s) \ K = V &= \{ Z_i(\mathcal{C}[i]) \mid i=1,\ldots,N \} \ P' &= \mathrm{softmax}\left(\frac{QK^\top}{\sqrt{d}} + M' \right)V . \end{split}$1 is the feature for view $\begin{split} Q &= Z_s(x_s, y_s) \ K = V &= \{ Z_i(\mathcal{C}[i]) \mid i=1,\ldots,N \} \ P' &= \mathrm{softmax}\left(\frac{QK^\top}{\sqrt{d}} + M' \right)V . \end{split}$2, and $\begin{split} Q &= Z_s(x_s, y_s) \ K = V &= \{ Z_i(\mathcal{C}[i]) \mid i=1,\ldots,N \} \ P' &= \mathrm{softmax}\left(\frac{QK^\top}{\sqrt{d}} + M' \right)V . \end{split}$3 is a blending ratio. The reported values are $\begin{split} Q &= Z_s(x_s, y_s) \ K = V &= \{ Z_i(\mathcal{C}[i]) \mid i=1,\ldots,N \} \ P' &= \mathrm{softmax}\left(\frac{QK^\top}{\sqrt{d}} + M' \right)V . \end{split}$4 for style edits and $\begin{split} Q &= Z_s(x_s, y_s) \ K = V &= \{ Z_i(\mathcal{C}[i]) \mid i=1,\ldots,N \} \ P' &= \mathrm{softmax}\left(\frac{QK^\top}{\sqrt{d}} + M' \right)V . \end{split}$5 for variable prompts such as character edits (Zhu et al., 15 Aug 2025).
The pipeline is motivated by a specific failure mode: without selective editing, inconsistent single-view results may be “averaged” into unnatural global artifacts. CoreEditor instead lets a selected reference impose stylistic direction while CCA enforces spatial consistency. A plausible implication is that the framework separates two constraints that are often conflated in multi-view editing: which visual mode to preserve and where cross-view agreement must hold.
5. Empirical evaluation
CoreEditor is evaluated against GaussianEditor, DGE, GaussCtrl, and EditSplat using CLIP similarity, CLIP directional similarity, Met3R multi-view consistency, and user study preference rates. The reported results are as follows (Zhu et al., 15 Aug 2025).
| Method | Text alignment | Consistency and preference |
|---|---|---|
| GaussianEditor | CLIP_sim 0.244; CLIP_dir 0.086 | User study quality 7.0%; consistency 8.8% |
| DGE | CLIP_sim 0.259; CLIP_dir 0.123 | Met3R 0.390; quality 14.6%; consistency 15.0% |
| GaussCtrl | CLIP_sim 0.257; CLIP_dir 0.128 | Met3R 0.372; quality 16.2%; consistency 14.8% |
| EditSplat | CLIP_sim 0.261; CLIP_dir 0.130 | Met3R 0.336; quality 17.0%; consistency 19.4% |
| CoreEditor | CLIP_sim 0.270; CLIP_dir 0.145 | Met3R 0.281; quality 45.2%; consistency 42.0% |
The paper reports best performance in all metrics for CoreEditor. Qualitatively, it states that competing methods often produce local artifacts, blurry details, or inconsistent edits when rendered from new views, whereas CoreEditor achieves stronger cross-view consistency and sharper edits. The supplemental video is described as showing that CoreEditor avoids flickering and detail loss in free-viewpoint renderings.
Ablation studies identify three components as essential. Removing CCA increases Met3R error and reduces CLIP scores. Using only geometry or only semantic correspondences is inferior to combining both. Removing reference attention causes inconsistent single-view results to be averaged into unnatural global artifacts. These ablations make the paper’s causal claim relatively specific: performance improvements are attributed to explicit cross-view constraints, correspondence co-support, and reference-guided selection rather than to a generic diffusion backbone.
6. Implementation profile, computational characteristics, and relation to prior methods
CoreEditor uses Splatfacto from Nerfstudio as the 3D representation and Stable Diffusion v1.5 with depth-conditioned ControlNet from the Diffusers library as the diffusion model. Joint editing uses $\begin{split} Q &= Z_s(x_s, y_s) \ K = V &= \{ Z_i(\mathcal{C}[i]) \mid i=1,\ldots,N \} \ P' &= \mathrm{softmax}\left(\frac{QK^\top}{\sqrt{d}} + M' \right)V . \end{split}$6 randomly sampled viewpoints, 500 GS optimization steps per scene, and 20 DDIM inversion/denoising steps. The optimization losses are L1 and LPIPS. For local edits, Lang-SAM is used for background exclusion. The reported resource profile is 18GB GPU and approximately 8 min per scene (Zhu et al., 15 Aug 2025).
The paper’s comparison table characterizes CoreEditor as having explicit cross-view constraints, geometry+semantic correspondence, user-driven style selection, no training/fine-tuning needed, and high efficiency. In contrast, the compared systems are marked as lacking explicit cross-view constraints, geometry+semantic correspondence, and user-driven style selection. This suggests that CoreEditor’s distinctiveness lies less in a new base generator than in the way it constrains information flow, constructs correspondences, and exposes user control at inference time.
Within the broader landscape of editor research, CoreEditor is notable because the term “editor” here refers neither to a text editor nor to a code editing agent, but to a diffusion-based 3D scene editing framework. Its central contribution is therefore not interface generation, collaborative text manipulation, or code transformation, but the introduction of correspondence-constrained diffusion for consistent text-to-3D editing. In that sense, CoreEditor occupies a specific place in the evolution of multi-view generative editing: it formalizes consistency as an attention-level constraint, supplements geometry with semantic evidence estimated during denoising, and uses selective reference propagation to align style across views.
