EditVerse3D: High-Quality 3D Object Editing with Region-Aware Learning

Abstract: Local editing of 3D objects remains a long-standing challenge. When interacting with 3D content, humans naturally tend to specify a coarse region of interest for modification rather than defining precise editing boundaries. However, previous methods rely on fully edited 2D images, precise 3D masks, or redundant pipelines, which present a gap. To bridge this gap, we propose EditVerse3D, a novel 3D editing framework that enables high-quality object editing under such coarse guidance. Our approach takes as input a 3D object to be edited, a coarse 3D bounding box indicating the target region, and a reference 2D image describing the desired modification. It produces a coherent, high-fidelity edited 3D object. To facilitate this editing, we introduce a novel region-aware adaptive loss that emphasizes hard-to-learn regions and balances the objective between target and preserved areas. Complementing our loss function, we enhance model robustness and generalization through targeted data augmentations, such as training with scaled 3D masks and filtering out unrealistic editing pairs. We construct a large-scale 3D editing dataset derived from parts information. Extensive experiments demonstrate that EditVerse3D achieves superior visual quality and quantitative performance compared to existing 3D editing approaches. Please visit our project page at https://editverse3d.github.io.

• The paper introduces an end-to-end rectified-flow model that edits 3D objects from a coarse bounding box and a single 2D reference image, avoiding 2D–3D reconstruction cycles and precise masks.
• Region-aware loss reweighting, bounding-box mask augmentation, and joint normalization improve coarse-region generalization, producing a Chamfer distance of 0.005 and PSNR of 28.67 on addition tasks.
• The method uses roughly 85,000 meshes and 500,000 editing pairs, runs in about 20 seconds, and substantially outperforms prior baselines while supporting both addition and replacement edits.

EditVerse3D addresses local 3D object editing under deliberately coarse user guidance: the model receives a 3D object, a coarse 3D bounding box marking the target region, and a single 2D reference image describing the desired edit, and directly produces an edited 3D asset end-to-end. The work is built on TRELLIS (Xiang et al., 2024), a rectified-flow 3D generative model, and contributes a supervised training strategy (region-aware loss reweighting, mask augmentation, joint normalization), a large-scale editing dataset of roughly 85k meshes and 500k pairs, and quantitative results that substantially outperform prior 3D–2D–3D and training-free editing pipelines.

Motivation and relation to prior work

Existing 3D editing approaches fall into three families, each with a structural weakness that EditVerse3D targets. Render-edit-lift methods (Chen et al., 2024, Qi et al., 2024, Labs et al., 17 Jun 2025) edit 2D renderings and reconstruct 3D from them, accumulating errors in the 3D–2D–3D round trip and suffering multi-view inconsistency; they also depend on the quality of 2D inpainting, which remains unreliable. Score-distillation methods (Yan et al., 2024, Gao et al., 2024, Han et al., 17 Apr 2025) optimize 3D representations under 2D priors but are computationally expensive, hard to optimize, and yield limited fidelity. Training-free methods built on 3D generative backbones (Li et al., 26 Aug 2025, Parelli et al., 29 Aug 2025, Ye et al., 16 Oct 2025) adapt 2D techniques such as RePaint (Graikos et al., 2022) and FlowEdit (Kulikov et al., 2024) to the 3D latent space, but generalize poorly beyond narrow cases and typically require pre-edited 2D views or precise 3D masks. The gap EditVerse3D identifies is that no prior method supports coarse, bounding-box-level guidance with an image prompt in a single end-to-end pass.

Architecture and training strategy

The method reuses TRELLIS's encoder–decoder, which compresses assets into a structure latent on a $64^3$ voxel grid and a texture latent over active voxels, and trains separate rectified flow models for editing. The input latents are concatenated channel-wise with a binary mask and noise; the conditioning is a rendering of the removed or target part rather than a fully edited 2D view, which removes the need for pre-edited images.

Three training design choices carry most of the empirical weight. First, region-aware loss reweighting: editing introduces an imbalance in which unedited regions converge easily while target regions carry persistently higher loss. The method normalizes masked and unmasked loss terms by region size so both contribute equally, and adds a hard-example term over the top-$\tau\%$ per-index losses, with terms balanced by magnitude. Second, mask augmentation: models trained with exact 3D masks converge on the training distribution but produce meaningless outputs when tested with coarse bounding boxes, so training uses minimal bounding boxes with random perturbations to box size and position. Third, joint normalization: the input object and mask must be normalized with a single shared factor; independent normalization distorts their relative spatial alignment, and the ablation shows the variant without joint normalization fails to converge entirely.

Dataset curation

Because no supervised 3D editing dataset existed for this input format, the authors synthesize editing pairs from part information: a part is removed from a complete object, the truncated object becomes the input, the original becomes ground truth, the removed part provides the 3D mask and is rendered to produce the 2D image prompt. Sources are the human-validated Partverse segmentation data [dong2025copart] and Objaverse assets with built-in part structure (Deitke et al., 2023), yielding ~85k meshes and ~500k pairs. The dataset consists almost entirely of "add" edits, but the paper argues—and demonstrates—that "add" is a special case of "replace" with an empty target region, and models trained on it generalize to replacement tasks. A notable ablation finding is that segmentation quality has negligible impact at fixed data scale: models trained on low-quality automatic part information match those trained on human-annotated segmentation, which is what makes the dataset scalable despite the scarcity of curated 3D segmentation data.

Experimental results

Evaluation uses a held-out set of ~200 meshes and ~1500 pairs from PartObjaverse-Tiny for addition, and the VoxHammer benchmark (100 meshes, 300 pairs) for replacement; geometry is scored with Chamfer distance and texture with PSNR, SSIM, LPIPS, DINO feature similarity, and FID over 32 rendered views, with metrics averaged over all 24 conditioning viewpoints to test robustness. Baselines are Instant3dit [bardaInstant3ditMultiviewInpainting2025], two re-implementations of TRELLIS editing via RePaint and FlowEdit, and VoxHammer (Li et al., 26 Aug 2025).

The quantitative margins are large. On the add task, EditVerse3D reaches CD 0.005, PSNR 28.67, and FID 2.960, versus 27.44/3.408 for the strongest baseline (RePaint) and 8.075/81.78 for Instant3dit, which performs worst by a wide margin. On replacement (evaluated on preserved regions, since ground truth for edited content is unavailable), the method attains CD 0.005 and PSNR 36.32. Region-wise analysis shows the advantage holds in both the edited region (PSNR 21.01 vs. 19.00 for RePaint) and the unedited region (PSNR 35.45, CD 0.002), indicating the loss reweighting preserves context without sacrificing edit fidelity.

The ablation isolates the contributions. Training with exact masks (CD 2.502×10⁻²) versus augmented bounding boxes (0.635×10⁻²) is the single largest factor, confirming the coarse-mask generalization problem. The region-aware loss, data filtering of implausibly small edit regions, and mask perturbations each add smaller but consistent gains. One counterintuitive result: randomly sampling one of 24 conditioning views per iteration outperforms deterministically traversing all views (CD 0.701 vs. 0.850), which the authors report without a mechanistic explanation. Robustness tests show the model tolerates 1.5× enlarged or shifted bounding boxes and image prompts from different viewing angles.

On efficiency, the method runs in ~20 seconds, matching vanilla TRELLIS and the Repaint/FlowEdit variants, versus 120 seconds for VoxHammer and 30 for Instant3dit, while requiring no 2D mask, no pre-edited 2D view, and only a coarse bounding box rather than an exact mask. This combination of lowest input burden and fastest runtime is the practical claim of the paper.

Limitations and open questions

Several constraints are stated or evident. The training data is dominated by "add" edits synthesized from part deletion; generalization to "replace" is demonstrated empirically on one benchmark but the distributional gap between synthesized restoration and genuine user-specified replacements is not characterized. The replacement evaluation lacks ground truth for edited regions, so edit fidelity on that task is assessed only qualitatively or on preserved regions. The VoxHammer comparison showed slightly worse results for that baseline than its reported performance, which the authors attribute to evaluation-configuration differences—an unresolved discrepancy. The hard-example mining parameter $\tau$ and the reason random view sampling beats traversal are left unexplained. Finally, conditioning uses a single randomly chosen 2D view at inference time; how performance degrades with ambiguous or conflicting image prompts is not studied.

Conclusion

EditVerse3D formulates local 3D editing as a supervised, end-to-end flow-matching problem under coarse bounding-box and image-prompt guidance, and supports it with a region-aware adaptive loss, mask augmentation, joint normalization, and a 500k-pair editing dataset. The results—roughly 5× lower Chamfer distance and 3–10× lower FID than the strongest training-free baselines at equal runtime and reduced input requirements—establish that supervised training on synthesized part-removal pairs is a viable route to practical 3D editing, and the dataset provides a foundation for subsequent supervised work in this area.