MVCHead: Single-Shot 3D Gaussian Head Avatars

• MVCHead is a single-shot state-space model that generates 3D head avatars using anisotropic 3D Gaussian splatting from unpaired 2D face images.
• It employs innovative HiSS and HiBiSS blocks to progressively upsample and refine geometric details while enforcing multi-view consistency.
• MVCHead achieves state-of-the-art results in appearance and geometric fidelity, outperforming previous 3D GANs on metrics like FID and Chamfer distance.

MVCHead is a single-shot, state-space based model for generating high-fidelity, multi-view consistent 3D Gaussian head avatars using only unpaired 2D images sampled randomly from datasets such as FFHQ and FFHQ-C, with no requirement for multi-view supervision, intermediate view synthesis, or external 3D priors. The method is notable for directly regressing the parameters of an anisotropic 3D Gaussian Splatting (3DGS) representation under architectural and loss constraints that enforce geometric and appearance consistency across views. Its core technical contributions are the Hierarchical State Space (HiSS) block for progressive, structure-aware upsampling and refinement, a novel Hierarchical Bi-directional State Scan (HiBiSS) mechanism to explicitly model view-drift, and a differentiable SE(3) Multi-view Critic that enables multi-view consistency rewards without access to synchronized image pairs (Chharia et al., 24 May 2026).

1. Problem Setting and Overview

MVCHead addresses the challenge of synthesizing 3D head avatars that remain consistent under varying viewpoints using only unpaired 2D images for supervision. This setting contrasts sharply with prior approaches that depend on (a) multi-view image sets, (b) explicit 3D scans, or (c) stepwise pipelines that first perform 2D view synthesis before lifting to 3D. By mapping a sampled latent code $z \sim \mathcal{N}(0,I)$ to a 3D Gaussian avatar in a single forward pass, MVCHead establishes a new paradigm for unconditional and conditional 3D head modeling.

Key distinctions of MVCHead include:

• Absence of multi-view data or pre-existing 3D face models during training.
• End-to-end architecture that regresses the fullest 3D representation directly from latent input.
• Multi-view consistency (MVC) enforced by both structural design and learning objectives.

2. Architectural Components

2.1 3D Gaussian Splatting Representation

MVCHead adopts the anisotropic 3D Gaussian Splatting (3DGS) format, wherein each of $N=240\,\mathrm{K}$ primitives $g_i$ is parameterized by:

• $\mu_i \in \mathbb{R}^3$: center
• $s_i \in \mathbb{R}^3_+$: axis-aligned scales
• $q_i \in \mathbb{H}$: unit quaternion (rotation $R(q_i) \in \mathrm{SO}(3)$)
• $\alpha_i \in (0,1)$: opacity
• $c_i \in [0,1]^3$: RGB color

Rendered images $I = R(S_\theta(z), T)$ are obtained via differentiable splatting under camera pose $N=240\,\mathrm{K}$0. Splatting computes local densities:

$N=240\,\mathrm{K}$1

with covariance $N=240\,\mathrm{K}$2. Final pixel colors are composited in depth order. Unlike NeRF-style methods, no explicit volumetric density or radiance field is modeled (Chharia et al., 24 May 2026).

2.2 Hierarchical State Space (HiSS) Block

HiSS blocks enable coarse-to-fine hierarchical modeling of head geometry and appearance. An input token grid $N=240\,\mathrm{K}$3 encodes features, processed by two complementary mixers:

• Self-Attention + MLP: captures global/non-axis-aligned dependencies (e.g. identity, lighting)
• State-space mixing via HiBiSS: propagates local, grid-aligned information, crucial for spatial coherence

Per-attribute MLPs regress new or refined Gaussian parameters, using anchor-offsets for finer levels. Upsampling at each HiSS stage allows local detail (e.g. facial features) to be iteratively refined, with structure-aware propagation that mitigates drift and fosters detail hierarchy (Chharia et al., 24 May 2026).

2.3 Hierarchical Bi-directional State Scan (HiBiSS)

While standard state-space models (Mamba/SSM) scan sequences unidirectionally, HiBiSS performs 2D scans along principal axes susceptible to view-drift:

• Horizontally: left→right and right→left
• Vertically: top→bottom and bottom→top

Recurrences of the form:

$N=240\,\mathrm{K}$4

are applied per direction and fused. The architectural bias leverages the fact that, for small yaw shifts $N=240\,\mathrm{K}$5, horizontal displacements dominate, and for pitch $N=240\,\mathrm{K}$6, vertical displacements dominate. This targeted smoothness and alignment yield lower view-to-view inconsistency (Chharia et al., 24 May 2026).

3. Multi-view Consistency Enforcement

3.1 SE(3) Multi-view Critic

The SE(3) Multi-view Critic $N=240\,\mathrm{K}$7 is a Vision Transformer-style encoder augmented with Geometric Transform Attention (GTA) for invariance to intrinsics/cropping and equivariance to global rigid transforms. It distinguishes between:

• Positive sets $N=240\,\mathrm{K}$8: views of the same latent $N=240\,\mathrm{K}$9 under different poses
• Negative sets $g_i$0: each $g_i$1 from a different latent $g_i$2 under the same pose

Training employs binary cross-entropy set classification:

$g_i$3

The generator receives the MVC reward:

$g_i$4

3.2 Full Training Objective

The total loss combines:

• $g_i$5
• Texture adversarial discriminators with R1 penalty
• Anchor center drift penalty $g_i$6
• $g_i$7-nearest-neighbor spacing regularization $g_i$8

Hyperparameters such as $g_i$9, $\mu_i \in \mathbb{R}^3$0, and $\mu_i \in \mathbb{R}^3$1 are used (Chharia et al., 24 May 2026).

4. Empirical Evaluation and Benchmarking

MVCHead exhibits state-of-the-art results on both appearance and geometric consistency metrics compared to previous feed-forward 3D GANs (GSGAN, GGHead, CGSGAN).

Metric	MVCHead	CGSGAN	GSGAN
FID (50K, 512²)	4.39	4.94	5.60
FID₃D (angles)	4.39	4.94	—
Chamfer (MVGBench/CD)	0.665	0.672	—
Geometric consistency (MEt3R)	0.262	0.281	—

Texture metrics (cPSNR, cSSIM, cLPIPS) favor MVCHead in all settings. Qualitatively, avatars display high-fidelity microgeometry (skin, wrinkles, hair wisps), and maintain stable silhouette and detail under large yaw/pitch, where prior methods show jitter or drift. Ablation studies confirm the necessity of both the HiBiSS mechanism and the differentiable consistency loss; removal of either degrades performance substantially (Chharia et al., 24 May 2026).

5. Training Methodology and Released Resources

Supervision uses only tight-cropped, single-view face portraits (FFHQ, FFHQ-C, 256K–70K images), with randomly sampled camera poses used for both generator rendering and critic evaluation. No multi-view or 3D-aligned data is required. Optimization is performed with Adam on 4 NVIDIA H100 GPUs for 10M steps (~3 days). Standard 2D augmentations are limited to random flips and color jitter.

The FaceGS-10K resource, released post-training, comprises 10,000 avatars, each rendered under 24 canonical viewpoints; the full collection contains 240K precomputed Gaussians and associated images, intended for further benchmarking and downstream evaluation, but not as training data for MVCHead itself (Chharia et al., 24 May 2026).

6. Limitations and Prospective Research

MVCHead is limited to the front-hemisphere (±90° yaw and pitch), lacking explicit modeling of back-of-head geometry and hair; extension to full 360° coverage would likely require advanced architectural priors or explicit occlusion completion. The current critic employs only “easy” negatives (different latents); employing “hard” negatives such as perturbed Gaussians of the same latent may further strengthen the multi-view consistency signal. All scene priors are learned from 2D supervision, suggesting that hybrid approaches incorporating explicit 3D priors could potentially enhance consistency and reduce learning complexity. Open research questions include multi-identity editing, explicit expression control, and integration with dynamic performance rigs (Chharia et al., 24 May 2026).