View-Consistent 3D Editing with Gaussian Splatting (2403.11868v10)

Abstract: The advent of 3D Gaussian Splatting (3DGS) has revolutionized 3D editing, offering efficient, high-fidelity rendering and enabling precise local manipulations. Currently, diffusion-based 2D editing models are harnessed to modify multi-view rendered images, which then guide the editing of 3DGS models. However, this approach faces a critical issue of multi-view inconsistency, where the guidance images exhibit significant discrepancies across views, leading to mode collapse and visual artifacts of 3DGS. To this end, we introduce View-consistent Editing (VcEdit), a novel framework that seamlessly incorporates 3DGS into image editing processes, ensuring multi-view consistency in edited guidance images and effectively mitigating mode collapse issues. VcEdit employs two innovative consistency modules: the Cross-attention Consistency Module and the Editing Consistency Module, both designed to reduce inconsistencies in edited images. By incorporating these consistency modules into an iterative pattern, VcEdit proficiently resolves the issue of multi-view inconsistency, facilitating high-quality 3DGS editing across a diverse range of scenes. Further video results are shown in http://vcedit.github.io.

• The paper presents VcEdit, a novel framework that eliminates multi-view inconsistencies and mode collapse in 3D Gaussian Splatting through innovative modules.
• It introduces a cross-attention module that aligns features across views, ensuring reliable diffusion-based guidance for high-fidelity 3D editing.
• The research provides robust code and demonstrations, enhancing the practical application and reliability of 3D Gaussian Splatting in real-world editing tasks.

"View-Consistent 3D Editing with Gaussian Splatting" introduces an advanced framework, View-consistent Editing (VcEdit), aimed at addressing the challenge of multi-view inconsistency in 3D Gaussian Splatting (3DGS). This inconsistency often results in mode collapse and visual artifacts when diffusion-based 2D editing models are used to modify multi-view rendered images.

3D Gaussian Splatting has become prominent for its ability to offer efficient and high-fidelity 3D rendering, making precise local manipulations more feasible. However, the integration of 2D editing models as a guiding mechanism for 3DGS models has been problematic due to discrepancies across different views. These differences culminate in visual flaws and unstable representations.

VcEdit is the proposed solution to maintain consistency across the multiple views rendered during the 3D editing process. This is achieved through the introduction of two innovative modules:

1. Cross-attention Consistency Module: This module is designed to ensure that attention mechanisms across different views remain aligned, thereby reducing inconsistencies during the editing process.
2. Editing Consistency Module: This module further stabilizes the editing process by continuously checking and maintaining consistency in the edited guidance images.

Together, these modules are employed iteratively within the VcEdit framework, dynamically adjusting and refining the edited outputs to ensure coherence and high-quality results across all views.

The paper demonstrates that VcEdit effectively mitigates the issues of multi-view inconsistency and mode collapse, leading to superior 3DGS editing outcomes across various scenes. Additionally, the authors have provided further resources, including code and video demonstrations, to showcase the capabilities and implementation of VcEdit. This contribution enhances the practicality and reliability of 3D Gaussian Splatting in real-world applications, offering a significant leap forward in the domain of 3D editing technology.