Edit3DGS: Dynamic 3D Head Editing

• Edit3DGS is a dynamic 3D head editing framework that transforms multi-view videos into editable, photorealistic avatars using 2D instruction-guided diffusion paired with 3D Gaussian Splatting.
• It features an iterative render-edit-aggregate pipeline that preserves identity, expression dynamics, and temporal coherence through mechanisms like masked latent inpainting.
• The framework integrates multi-view consistency and FLAME-driven motion control, making it suitable for applications in virtual avatars, immersive media, and AR/VR.

Edit3DGS is a framework for dynamic 3D head editing that combines 2D instruction-guided diffusion editing with 3D Gaussian Splatting (3DGS) to produce an editable, photorealistic, temporally coherent head avatar from a multi-view video. It targets expression transformation, attribute modification, and appearance refinement, while aiming to preserve both identity and motion dynamics. The method is organized as an iterative render-edit-aggregate pipeline: rendered images from a dynamic Gaussian head avatar are edited in 2D under text guidance, then re-aggregated into an updated 3D representation through Gaussian fitting (Tran et al., 16 Jun 2026).

1. Task setting and conceptual position

Edit3DGS addresses a problem that is more constrained and more dynamic than most earlier 3DGS editing formulations: the input is a multi-view video of a person’s head, and the output is a 3D avatar that remains animatable after semantic editing. The framework is motivated by two limitations. First, frame-based 2D editing can follow text instructions, but if applied independently to each frame or view it introduces temporal flicker, multi-view inconsistency, identity drift, and unstable 3D reconstruction. Second, existing static 3D editing methods and reconstruction-only 3D head avatar methods are not designed for direct semantic editing under dynamic facial motion (Tran et al., 16 Jun 2026).

The method is therefore positioned at the intersection of two research lines. On one side are 3DGS editing systems for static scenes or objects. For example, DGE formulates editing of an already reconstructed 3D scene or object represented with 3DGS as a problem of first producing multi-view consistent edited views and then directly fitting the Gaussian scene to them (Chen et al., 2024). On the other side are dynamic head avatar models such as GaussianAvatars, FlashAvatar, and MonoGaussianAvatar, which emphasize photorealistic rendering and animation rather than editable semantics (Tran et al., 16 Jun 2026). Edit3DGS combines these directions by adapting multi-view-consistent 2D editing to a dynamic head avatar setting and by adding an explicit mechanism for preserving expression cues across timesteps.

A central feature of the framework is that it starts from an identity-specific GaussianAvatars model rather than reconstructing a head from scratch. This choice makes the method a per-subject, per-scene optimization framework: the subject’s head geometry, appearance, and FLAME-driven motion model are already present, and the task is to inject text-conditioned semantic changes without breaking the avatar’s animatability or temporal coherence (Tran et al., 16 Jun 2026).

2. Pipeline architecture

The pipeline begins with a GaussianAvatars model trained on a multi-view head video dataset. In this base representation, a set of 3D Gaussians is fitted to the head and rigged to the FLAME mesh triangles, which provides a dynamic, animatable, photorealistic head avatar. Edit3DGS uses this model as its starting point (Tran et al., 16 Jun 2026).

From the current Gaussian avatar, the system renders a set of images from selected camera views and selected timesteps. These renders are not treated as final outputs but as an editable 2D dataset. The method then identifies facial regions whose preservation is especially important for expression dynamics, focusing on the eyes and mouth. For mask extraction, the implementation combines SAM 2 and Grounding DINO (Tran et al., 16 Jun 2026).

The image editing stage uses Instruct-Pix2Pix as the instruction-guided diffusion model. However, the images are not edited independently. At each timestep, the corresponding rendered views are treated as a batch, edited jointly, and then aggregated into an edited multiview dataset. This edited dataset becomes pseudo-ground-truth supervision for updating the 3D Gaussian avatar. The update step is described as Edit Through Gaussian Fitting: if edited 2D supervision is sufficiently coherent across views and time, then the underlying Gaussian head model can be refit directly to match it (Tran et al., 16 Jun 2026).

The loop can then be repeated. After a refined edited avatar has been obtained, it can be rendered again, reprocessed through the 2D editing stage, and used to propagate changes to previously unseen timesteps. This iterative structure is one of the framework’s defining properties, because it treats 2D diffusion not as an end in itself but as a means of constructing reconstructable supervision for a dynamic 3D representation (Tran et al., 16 Jun 2026).

3. Spatial and temporal consistency mechanisms

Edit3DGS separates consistency into spatial/multi-view consistency and temporal/expression consistency. The spatial component is adapted from DGE. At a fixed timestep $t$, the set of rendered views $V$ is input to the diffusion model as a batch. Editing proceeds in two substeps: key-view editing and feature injection. A random subset of views is selected as key views; these are edited jointly using spatial-temporal attention blocks, allowing each view to incorporate contextual information from the others. The resulting edited features are then propagated to the remaining views using correspondences established by epipolar constraints and matching visual features across denoising layers (Tran et al., 16 Jun 2026).

This design addresses the specific failure mode of independent per-view editing, where age changes, wrinkles, makeup, or other facial details may appear differently across camera angles. The paper frames this spatial strategy explicitly as being inspired by DGE, whose core insight is that direct Gaussian fitting becomes practical once multi-view-consistent edited images are available (Chen et al., 2024).

Temporal consistency is handled differently. The authors observe that text-guided diffusion editors often produce nearly identical faces at the same viewpoint across different timesteps, which suppresses complex expressions and weakens facial dynamics. The most affected regions are the eyes and mouth. To counter this, Edit3DGS introduces lightweight masked latent inpainting. Input images are mapped into latent space at different noise levels; masks isolate the eye and mouth regions; the latent vectors from the original images are extracted; and these latents are re-injected during later denoising steps. The intended effect is to preserve gaze, lip motion, and subject-specific expression cues, rather than letting the diffusion model wash them out into a more neutral face (Tran et al., 16 Jun 2026).

The paper does not formalize these mechanisms with explicit new consistency losses. Instead, spatial consistency is enforced procedurally through batched multi-view editing with feature propagation, while temporal consistency is enforced through masked latent reuse in expression-critical regions. This suggests that the framework’s novelty lies less in a new analytic objective than in the orchestration of existing 2D and 3D components into a dynamic editing loop.

4. Representation, fitting, and optimization

The underlying 3D representation is the GaussianAvatars head model, which itself is built on standard 3D Gaussian Splatting and FLAME-guided deformation. The paper does not enumerate the Gaussian parameter vector explicitly, but it states that GaussianAvatars fits a set of 3D Gaussian splats to a FLAME mesh and associates each Gaussian with a parent triangle on the mesh. This provides animatability through FLAME, view-consistent appearance, efficient rendering, and photorealism (Tran et al., 16 Jun 2026).

Edit3DGS does not introduce a new 3DGS renderer or a new Gaussian parameterization. Conceptually, the fitting process is straightforward: render the current avatar, edit those renders in 2D, and then re-optimize the Gaussian model so that its rendered outputs match the edited images. The practical optimization setup is explicitly reported: the implementation uses the Adam optimizer, a learning rate of $1e-2$, and $10 \times n$ iterations per editing pass, where $n$ is the number of timesteps or batches in the considered dataset (Tran et al., 16 Jun 2026).

A notable characteristic of the method description is that the paper does not provide explicit mathematical formulas for Gaussian rendering, diffusion denoising, multi-view consistency loss, temporal consistency loss, inpainting objective, or a full total optimization objective. The method is described procedurally rather than through a newly formalized analytical loss. This is consistent with the paper’s emphasis on system integration: standard 3DGS and GaussianAvatars are reused, while the main novelty is the dynamic editing pipeline wrapped around them (Tran et al., 16 Jun 2026).

The final output remains an editable dynamic 3D avatar. After fitting, it supports novel-view rendering, novel pose and expression control through FLAME parameters, self-reenactment, and cross-identity reenactment. In this sense, Edit3DGS is not merely a renderer-side appearance editor; it is a semantic editing layer over an animatable head representation (Tran et al., 16 Jun 2026).

5. Evaluation and empirical profile

The experiments use the NeRSemble dataset, described as a multi-view video dataset of human heads with 11 video sequences per subject, each sequence containing approximately 150 frames. Images are downscaled to $802 \times 550$. Following GaussianAvatars, the evaluation protocol leaves one video sequence and one camera view for quantitative evaluation (Tran et al., 16 Jun 2026).

The primary baseline is GaussianAvatar-Editor (also denoted GA-Editor), which the paper identifies as the only prior method addressing editable 3DGS-based dynamic head models. Evaluation is performed in three settings: novel view rendering, self-reenactment, and cross-identity reenactment. The reported metrics are CLIP Text-Image Direction Similarity (CLIP-S) and CLIP Direction Consistency (CLIP-C) (Tran et al., 16 Jun 2026).

Protocol	GA-Editor	Edit3DGS
Novel view CLIP-S	0.258	0.269
Novel view CLIP-C	0.978	0.969
Self-reenactment CLIP-S	0.076	0.071
Self-reenactment CLIP-C	0.954	0.957
Cross-identity reenactment CLIP-S	0.072	0.074
Cross-identity reenactment CLIP-C	0.965	0.945

The numerical gaps are described as small, and both methods are characterized as highly competitive. The authors emphasize that the main difference is qualitative rather than purely metric: Edit3DGS is reported to preserve complex facial expressions, wrinkles, eye gaze, and lip motion more effectively under temporally challenging sequences (Tran et al., 16 Jun 2026).

The qualitative experiments illustrate three recurring use cases. In novel view rendering, a prompt such as “Make him look older” yields age-related changes while preserving identity and structure. In self-reenactment, the edited avatar is driven by unseen expressions and poses from the same actor while maintaining temporal coherence. In cross-identity reenactment, expressions from another actor are transferred while preserving the edited target’s identity (Tran et al., 16 Jun 2026).

The principal ablation concerns the inpainting mechanism. Without inpainting, edited views fail to retain the original expressions, and details around the eyes and mouth degrade, which in turn worsens animation quality. This ablation supports the claim that masked latent inpainting is not merely cosmetic but structurally important for preserving dynamic facial behavior (Tran et al., 16 Jun 2026).

6. Limitations, applications, and relation to the broader 3DGS editing literature

The paper identifies several limitations. Because the representation inherits GaussianAvatars and FLAME, fine details such as teeth and hair are not perfectly captured. Performance also depends on the chosen 2D diffusion editor, here Instruct-Pix2Pix, so editing speed, stability, and fidelity vary with the backbone. The framework is naturally sensitive to text prompt quality, to the accuracy of eye and mouth masks, and to difficult dynamic regimes such as large pose changes, unusual expressions, and out-of-distribution edits. It also assumes a reasonably good multi-view head capture and a valid initial GaussianAvatars reconstruction; it is not presented as a monocular casual editing system (Tran et al., 16 Jun 2026).

The application profile follows directly from these design choices. The method is intended for virtual avatars, immersive communication, film and gaming pipelines, and AR/VR or interactive media. Its specific practical appeal lies in producing an edited representation that remains animatable rather than a collection of edited frames: the output can still be rendered from new views and driven by new FLAME parameters after semantic modification (Tran et al., 16 Jun 2026).

Within the broader 3DGS editing literature, Edit3DGS occupies a distinctive niche. GSEdit focuses on text-guided editing of a 3D object already represented as, or convertible to, 3DGS, using Instruct-Pix2Pix and an SDS gradient, with emphasis on object appearance, style, and moderate identity or shape changes (Palandra et al., 2024). DGE frames static 3DGS editing around the generation of multi-view consistent edited views followed by direct Gaussian fitting (Chen et al., 2024). Edit3DGS inherits the static multi-view-consistency logic of DGE, but extends it to dynamic heads by integrating FLAME-based avatar control, batch editing across views, and latent inpainting across timesteps (Tran et al., 16 Jun 2026).

A plausible implication is that Edit3DGS marks a transition from static 3DGS editing toward editable dynamic avatars. Its contribution is not a new Gaussian rendering equation, nor a new diffusion backbone, but a unified system that makes instruction-guided editing compatible with temporal coherence, identity preservation, and animation-ready 3D head models.