Edit3DGS: Unified Framework for Dynamic Head Editing via 2D Instruction-Guided Diffusion and 3D Gaussian Splatting

Abstract: We present Edit3DGS, a unified framework for dynamic 3D head editing that integrates 2D instruction-guided diffusion with 3D Gaussian splatting. Unlike prior approaches that separately address frame-based edits or static 3D reconstruction, our method couples semantic controllability in the image domain with photorealistic, temporally consistent 3D representations. Given an input video, editable facial regions are masked and modified using a text-conditioned diffusion model to support fine-grained operations such as expression transformation, attribute modification, and appearance refinement. The edited frames are then aggregated through 3D Gaussian splatting to produce a coherent, high-fidelity avatar that preserves both identity and motion dynamics. To enforce consistency, Edit3DGS incorporates multi-view batch editing and lightweight inpainting strategies that recover lost expressions across timesteps. Experimental results demonstrate that our framework enables controllable, artifact-free head editing with smooth temporal transitions, offering practical applications in virtual avatars, immersive communication, film production, and interactive media.

• The paper introduces Edit3DGS, a novel framework that combines 2D diffusion and 3D Gaussian Splatting for dynamic head editing.
• It employs multi-view batch editing and masked inpainting to maintain spatial-temporal consistency and preserve facial expression details.
• Experimental results demonstrate robust semantic alignment and seamless edit propagation, enhancing avatar realism in AR/VR and interactive media.

Edit3DGS: Unified Framework for Dynamic Head Editing via 2D Instruction-Guided Diffusion and 3D Gaussian Splatting

Introduction and Motivation

This paper introduces Edit3DGS, a framework explicitly developed for dynamic, high-fidelity 3D head editing by integrating instruction-guided 2D diffusion models with 3D Gaussian Splatting (3DGS) representations (2606.17432). The key motivation is to address the significant gap in existing avatar editing pipelines: most state-of-the-art 3DGS approaches focus on efficient reconstruction and animation but lack robust and semantically controllable editing capabilities. Meanwhile, diffusion-based models excel in intuitive and granular image manipulation via textual prompts, but direct extension to volumetric editing introduces spatial-temporal inconsistency and artifacts, especially with dynamic facial expressions.

Edit3DGS leverages multi-view batch editing and mask-based refinement strategies to enforce spatial coherence and preserve expression dynamics across frames, achieving photorealistic, temporally consistent edits suitable for digital avatars in AR/VR, film, and interactive communication.

Figure 1: Overview of the Edit3DGS framework depicting multi-view batch editing and mask-based refinement to ensure spatial/temporal consistency and fine-grained controllability.

Technical Approach

Gaussian Splatting Avatar Backbone

The system foundation is the GaussianAvatars pipeline, which rigs 3D Gaussians to FLAME parametric mesh, jointly optimizing geometrical and appearance parameters to capture expression, pose, and identity with high fidelity. Gaussian primitives are distributed across mesh triangles, yielding precise animation control and superior photorealism compared to NeRF-based alternatives. The FLAME model remains a constraint, predominantly modeling skin and facial anatomy, while omitting non-explicit regions like hair and teeth.

Edit Propagation via Batch-Based 2D Diffusion

2D image editing is performed by text-conditioned diffusion models (Instruct-Pix2Pix) applied over rendered views from different camera poses and timesteps. Standard independent frame editing yields inconsistent results; to mitigate this, Edit3DGS extends multi-view batch editing inspired by DGE [chen2024dge]. Views are edited in groups (batch input), with spatial-temporal attention and feature injection across key views to capture mutual dependencies, enforcing both inter-view coherence and intra-frame consistency.

Temporal Consistency and Masked Inpainting

Temporal consistency across dynamic sequences is established by iterative render-edit-aggregate cycles, adapting principles from Instruct-NeRF2NeRF [haque2023instruct]. However, generic diffusion editors erase nuanced facial expressions. Edit3DGS introduces lightweight latent inpainting: auto-generated masks (via SAM 2 and Grounding DINO [ravi2024sam, liu2024grounding]) isolate latent features in critical expression regions (eyes, mouth). These are reintegrated into the denoising process, preserving original emotion and local dynamism for each frame.

Gaussian Fitting for Model Update

Edited multi-view renders are aggregated, and the original 3DGS structure is updated through Gaussian fitting. This direct update pipeline leverages the efficiency and geometry expressiveness of 3DGS, bypassing voxel-based optimization. Iterative editing allows propagation to previously unseen poses and expressions, facilitating flexible, high-quality avatar manipulation.

Experimental Results

Qualitative Evaluation

The NeRSemble dataset is utilized for multi-view, multi-expression quantitative analysis. Edit3DGS demonstrates strong spatial and semantic consistency across tasks:

• Novel View Synthesis: Textual attribute edits (aging, facial hair) render consistently across multiple viewpoints, preserving geometric identity and structural details.
• Self-Reenactment: The edited avatar tracks unseen expressions and poses from the same actor, maintaining smooth transitions and expression fidelity.
• Cross-Identity Reenactment: The system accurately transfers complex motion dynamics between different subjects while preserving target identity, addressing the challenge of balancing expression transfer and identity stability.

  Figure 2: Consistent and high-quality edits across viewpoints for the prompt "Make him look older", with identity preservation in novel view synthesis.

  

  Figure 3: Self-reenactment qualitative results showing smooth edit propagation and retention of subtle expression details on unseen actor poses.

  

  Figure 4: Cross-identity reenactment results illustrating accurate transfer of expressions and pose between different actors.

Quantitative Metrics

Metrics employed include CLIP Text-Image Direction Similarity (CLIP-S) and CLIP Direction Consistency (CLIP-C), comparing semantic accuracy and edit directionality. Edit3DGS achieves:

• Higher CLIP-S scores in novel view and cross-reenactment tasks, indicating improved semantic alignment with prompts.
• Comparable CLIP-C scores to GaussianAvatar-Editor [liu2025gaussianavatar], confirming edit consistency.

Absolute score differences between Edit3DGS and the state-of-the-art baseline remain narrow, underscoring matched performance.

Ablation: Impact of Inpainting

Disabling inpainting results in loss of critical expression details, particularly around the eyes and mouth, leading to dramatic degradation of animation quality. Masked inpainting proves essential for sustaining local dynamism and fine-grained editability.

Figure 5: Comparison between edited results with and without inpainting, highlighting loss of expression fidelity in the absence of the mask-based refinement.

Practical and Theoretical Implications

Edit3DGS bridges semantic-driven 2D editing and photorealistic, dynamic 3D avatar construction, supporting practical deployment in avatar-driven communication, virtual assistants, film asset creation, and interactive media. The batch-wise multi-view diffusion and latent inpainting pipeline enables fine-grained control, temporal stability, and resilience against geometry-attribute mismatches, marking methodological progress toward more expressive and editable volumetric representations.

Theoretically, the integration strategy underscores the importance of spatial-temporal attention and feature sharing for 3D consistency, advancing understanding of multi-view editing dynamics. Masked inpainting introduces a flexible approach for expression retention in generative pipelines, with potential extension to non-facial and non-rigid domains.

Future directions include integration with more expressive 3D priors, enhancement of non-explicit regions modeling, adaptive prompt handling, and domain transfer across diverse avatar formats.

Conclusion

Edit3DGS sets a unified and efficient technical paradigm for dynamic 3D head editing, combining multi-view 2D diffusion, spatial-temporal attention, and masked inpainting, enabling artifact-free, semantically controllable avatar edits. The framework's performance is competitive with the state-of-the-art and displays practical advantages in edit granularity and temporal stability. Its limitations, stemming largely from underlying priors and diffusion editors, motivate avenues for further research in robust avatar modeling and expressive generative pipelines.