AvatarBack: Complete Avatar Presence

• AvatarBack is a research concept that restores missing human presence in avatars through techniques like reverse pass-through VR and 3D back-head completion.
• The framework employs multi-view synthesis, adaptive alignment, and diffusion-based inpainting to reconstruct facial and back features in real time.
• It also integrates editing and trust back-end methods to ensure consistent, socially legible, and secure avatar communications.

AvatarBack is a research term used across several adjacent literatures on avatars, telepresence, and VR to denote methods that restore missing human presence where conventional capture or display pipelines fail. In current usage, it can refer to restoring a VR user’s facial presence to collocated people by outward rendering on a headset, reconstructing complete back-head geometry for 3D Gaussian head avatars, reconstructing the back side of clothed human avatars from monocular in-the-wild videos, deploying a universal avatar-editing back-end for heterogeneous volumetric representations, or supporting auditing and authorized-use verification for synthetic talking-head systems (Dash et al., 24 May 2025, Xin et al., 28 Aug 2025, Guo et al., 3 Mar 2025, Bao et al., 2024, Prashnani et al., 2023).

1. Terminological scope

The term does not denote a single canonical architecture. Instead, it appears in multiple technically distinct settings that share a common objective: recovering, rendering, editing, or verifying aspects of human presence that are absent, occluded, or weakly supervised. In one line of work, AvatarBack means “restoring the user’s facial presence to collocated people while the user is in VR—by projecting reconstructed 2D faces or animating full 3D head avatars on the outward-facing display of the headset.” In another, AvatarBack is “a novel plug-and-play framework specifically designed to reconstruct complete and consistent 3D Gaussian avatars by explicitly modeling the missing back-head regions.” A third usage explains “how Vid2Avatar-Pro reconstructs and renders the back side of human avatars from monocular in-the-wild videos.” A fourth treats GeneAvatar as a universal avatar-editing back-end. A fifth frames AvatarBack as a trustworthy-avatar stack that supports “auditing/backtracking of expression sources” (Dash et al., 24 May 2025, Xin et al., 28 Aug 2025, Guo et al., 3 Mar 2025, Bao et al., 2024, Prashnani et al., 2023).

Paper	AvatarBack usage	Technical core
"Eye-See-You: Reverse Pass-Through VR and Head Avatars" (Dash et al., 24 May 2025)	Restoring facial presence to collocated people in VR	CycleGAN alignment, cross-attention GAN restoration, one-shot 3D head avatar
"AvatarBack: Back-Head Generation for Complete 3D Avatars from Front-View Images" (Xin et al., 28 Aug 2025)	Completing missing rear head regions in 3D Gaussian avatars	Subject-specific Generator and Adaptive Spatial Alignment Strategy
"Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Prior" (Guo et al., 3 Mar 2025)	Recovering the back side of human avatars from monocular videos	Universal Prior Model, canonical front/back maps, diffusion-based texture inpainting
"GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image" (Bao et al., 2024) / "Avatar Fingerprinting for Authorized Use of Synthetic Talking-Head Videos" (Prashnani et al., 2023)	Editing back-end and trust back-end	Expression-aware 3D modification field / appearance-invariant motion signatures

Taken together, these usages suggest that AvatarBack functions less as a proper noun for one model than as a cluster of objectives around recovering complete, socially legible, and controllable avatar presence.

2. Reverse pass-through VR and outward facial presence

In reverse pass-through VR, the central problem is that occluded eyes and upper face break critical social signals—gaze, eye contact, and micro-expressions—so VR usage becomes isolating in shared spaces. RevAvatar addresses this with a complete reverse pass-through framework that reconstructs the user’s occluded face and eyes from partial VR observations and drives a one-shot 3D head avatar, both in real time, to convey eye contact, expressions, and nonverbal cues to the physical world (Dash et al., 24 May 2025).

The initial setup uses a Digital Persona (DP): the user captures or uploads a selfie via the headset’s external camera or account during setup, and the DP is processed by a landmark detector for cropping and alignment. The DP anchors identity for both 2D restoration and 3D avatar generation without requiring multi-view scans or personalized capture rigs. Runtime inputs are non-aligned left and right eye patches, often grayscale IR with occlusions and narrow FOV, plus a lower-face region. Three CycleGAN-based alignment and frontalization networks are used: CG_LE for left eye, CG_RE for right eye, and CG_Face for lower face. Bidirectional mapping between tilted and frontal eye views is learned via cycle consistency, and the alignment stage reports gaze estimation error below 1 degree between aligned and ground-truth eyes.

The 2D restoration stage uses a lightweight GAN with Generator $G$ and a Multiscale PatchGAN Discriminator $D$. The generator has two encoders: Input Encoder $E_I$, which consumes the DP with pasted partial observations $x$, and Reference Encoder $E_R$, which consumes the reference DP $z$. Cross-attention at multiple scales fuses reference identity and context with the input’s partial observations, enabling reconstruction of occluded areas such as eyebrows while preserving identity and expression. The decoder synthesizes the restored full-face output $G(x,z)$. The training objective is reported as

$$\mathcal{L}_{G} = \lambda_{adv} \mathcal{L}_{adv} + \lambda_{L1} \mathcal{L}_{L1} + \lambda_{LPIPS} \mathcal{L}_{LPIPS}$$

with

$$\mathcal{L}_{adv} = \mathbb{E}_{y} \left[ \log D(y) \right] + \mathbb{E}_{x} \left[ \log(1 - D(G(x, z))) \right].$$

For AvatarBack deployment, the eye region or full face is cropped to match the outward display’s ROI.

The 3D branch uses one-shot head avatar generation. The source image $I_s$ is the user’s DP, which supplies identity, and the target image $D$0 is the restored 2D face, which supplies current pose and expression. A Global Branch $D$1 based on a hybrid convolution-transformer plus SegFormer maps 2D inputs to a canonical 3D tri-plane $D$2 representing neutral identity geometry, regularized by a 3D Morphable Model. A Detail Branch $D$3 refines $D$4 into an appearance tri-plane $D$5. An Expression Branch $D$6 estimates 3DMM shape and expression coefficients for $D$7 and $D$8, renders a frontal expression image $D$9 from $E_I$0, and encodes it to an expression tri-plane $E_I$1. The final tri-plane is $E_I$2, followed by low-resolution volumetric rendering and a GFP-GAN-based super-resolution module.

VR-Face underpins both training and evaluation. It contains 200,000 samples; each sample includes a full-face image, left eye, right eye, and lower face. Viewpoints are front, left-60°, right-60°, and top-30°. Expressions are anger, contempt, disgust, fear, happy, neutral, sad, and surprise. Preprocessing effects include distortion, masking, vignetting, blur, grayscale, occlusions, lighting shifts, noise, and eyebrow reduction. The dataset is designed to emulate commercial HMD conditions and is presented as public.

Quantitatively, the 2D model reports, for the full face, SSIM 0.9445, PSNR 31.3951, LPIPS 0.0243, and 0.006 s inference time; for the eye ROI, SSIM 0.8572, PSNR 28.2897, and LPIPS 0.0510. DifFace reports higher SSIM at 0.9541 but lower PSNR at 29.8129, higher LPIPS at 0.1306, and 6.125 s latency. For one-shot 3D head avatars, RevAvatar reports SSIM 0.8025, PSNR 25.1284, and LPIPS 0.0629, compared with Portrait4D-v2 at 0.7922, 24.3271, and 0.0638. Component latencies are alignment $E_I$3 s, face restoration $E_I$4 s per image, and avatar rendering $E_I$5 FPS on a single A100. Measured full-face restoration latency is 0.006 s on A100, 0.012 s on Apple M2, 0.125 s on MX350, and 0.0517 s on GTX 1050.

Practical deployment still requires calibration of display ROI position and scale on the headset shell, camera-to-display mapping for gaze alignment, and per-device fine-tuning for camera baselines and outward display geometry. The paper does not present formal user studies; qualitative visuals and quantitative metrics suggest improvements in social presence capability, but subjective measures and statistical tests are not reported.

3. Back-head completion in 3D Gaussian head avatars

A second, more geometrically focused meaning of AvatarBack concerns the back-head failure mode in 3D Gaussian Splatting head avatars trained from frontal-view images. When training data emphasize the frontal hemisphere, rear regions become unobserved and unsupervised, producing geometric inconsistencies, structural blurring, reduced realism, and animatability failures under 90–180° azimuths. The 2025 AvatarBack method addresses this by explicitly modeling the missing back-head regions while remaining plug-and-play with GaussianAvatars and SurfHead (Xin et al., 28 Aug 2025).

The representation follows FLAME-driven Gaussian head avatars. Each Gaussian is attached at a triangle center and parameterized as

$E_I$6

with mean position, anisotropic scale, rotation, opacity, and view-dependent color. FLAME parameters $E_I$7 provide identity, expression, and pose, and the bound Gaussians inherit motion from the animatable mesh. Rendering is abstracted by a differentiable Gaussian renderer $E_I$8.

AvatarBack introduces two core components. The Subject-specific Generator (SSG) is a geometry-aware generative prior built on PanoHead. It is adapted per subject by hybrid multi-view inversion that merges captured frontal images with 3DGS-rendered novel views from the current avatar. After adapting generator weights $E_I$9 and latent code $x$0, it synthesizes pseudo back views for azimuths in $x$1, then sharpens them with GCFSR. The Adaptive Spatial Alignment Strategy (ASA) learns a transformation applied to FLAME vertices, and therefore to the bound Gaussians, to resolve coordinate discrepancies between pseudo-images and the 3DGS avatar. The learnable transformation is decomposed into scale, rotation, and translation:

$x$2

ASA is co-optimized with Gaussian attributes and FLAME parameters, while a FLAME regularizer

$x$3

keeps mesh parameters close to the original solution.

Training mixes real frontal images at 802×550 and SSG-generated back pseudo-images at 512×512. The pseudo-image supervision weight is $x$4, the FLAME regularization weight is $x$5, and Adam uses an initial learning rate 0.005 for ASA parameters with cosine annealing. Hardware is 4× NVIDIA RTX 3090 Ti (24 GB each). Inference is unchanged: the completed avatars remain real-time and fully animatable.

Evaluation uses NeRSemble and K-hairstyle. On K-hairstyle, GaussianAvatars improves from FID 218.34 and KID 0.202 to 146.73 and 0.120 with AvatarBack; SurfHead improves from 232.46 and 0.227 to 165.06 and 0.146. GPT-4o-based perceptual scores for the back region improve from 6.40 to 8.20 for GaussianAvatars and from 6.73 to 8.43 for SurfHead. On NeRSemble frontal metrics, GaussianAvatars+AvatarBack improves Novel-view PSNR 31.6 → 31.8, SSIM 0.938 → 0.939, and LPIPS 0.065 → 0.064; SurfHead+AvatarBack shows larger gains, with Novel-view PSNR 30.07 → 32.75, SSIM 0.934 → 0.940, and LPIPS 0.079 → 0.069.

The ablation studies isolate the two proposed components. SSG alone improves mesh completeness and back geometry but still lacks full regularity. ASA with FLAME regularization enforces geometry regularity, but without reliable back supervision rear geometry remains implausible. The full system—SSG + ASA + $x$6—produces smooth surfaces, anatomically plausible back-head structure, and consistent geometry across 0°, 90°, 180°, and 270° views. The paper also notes that PanoHead reaches a GPT-4o perceptual score of 6.94, higher than GaussianAvatars but lower than AvatarBack-enhanced methods, underscoring that multi-view synthesis alone cannot ensure geometry-consistent, animatable reconstructions.

4. Monocular avatar reconstruction and learned priors for unseen backs

A broader AvatarBack problem arises when only monocular or frontal-biased video is available. Vid2Avatar-Pro addresses this at full-body scale by learning a Universal Prior Model from a large corpus of multi-view clothed human performance capture data, then fine-tuning that prior on a monocular in-the-wild video of an unseen identity (Guo et al., 3 Mar 2025).

The pipeline has two stages. First, the UPM is trained on 1,000 identities, captured with 200 cameras at 4096×2668 resolution and approximately 5,000 frames per person. The model uses canonical front and back maps as a universal parameterization of clothed humans. From a reconstructed canonical textured template, the system orthogonally projects front and back canonical position maps, front and back canonical texture maps, and canonical template masks. The UPM consumes identity-specific canonical textures together with posed position maps and predicts pose-dependent Gaussian maps:

$x$7

Second, for in-the-wild personalization, the method estimates SMPL-X-style pose and shape, segments the person, reconstructs a canonical textured template, and uses a diffusion-based inpainting module to jointly complete missing front and back textures before inverse-rendering fine-tuning. The training and personalization loss is

$x$8

The back-side reconstruction mechanism is explicit: the UPM learns plausible back-side geometry and appearance from truly multi-view captures; the canonical front/back maps make back texels spatially aligned across identities; diffusion-based canonical texture inpainting fills unseen back regions; and fine-tuning aligns the prior to the target identity without overfitting to front views. On NeuMan interpolation, Vid2Avatar-Pro reports PSNR 32.71, SSIM 0.983, and LPIPS×100 1.19, compared with ExAvatar at 31.39, 0.981, and 1.64. On MonoPerfCap extrapolation, it reports 31.97, 0.981, and 1.37, compared with ExAvatar at 30.29, 0.979, and 2.19. The inpainting ablation is quantitatively modest but qualitatively crucial: PSNR 30.17 → 30.22 and LPIPS×100 2.22 → 2.18.

GAN-Avatar addresses a related failure mode for head avatars from commodity imagery: prior monocular pipelines depend on face tracking and landmark detection, which fail for profile and back views, so training losses break down for ears, hair, and the back of the head. GAN-Avatar removes the dependency on facial expression tracking entirely when learning appearance, allowing training from all viewpoints for which images and camera parameters are available (Kabadayi et al., 2023).

Its backbone is EG3D with tri-plane representation, initialized from FFHQ and fine-tuned on a single subject as an unconditional 3D-aware appearance model. The model uses known camera parameters, Procrustes alignment instead of landmark-based normalization, and higher ray sampling within a head-bounded volume. Control is decoupled: a shallow MLP $x$9 maps expression parameters $E_R$0 to latent codes $E_R$1 using synthetic frontal render pairs, while the personalized generator remains frozen. On NeRFace and INSTA self-reenactment, GAN-Avatar reports MSE 0.0023, PSNR 27.44, SSIM 0.91, and LPIPS 0.06. On Multiface v2 frontal-hemisphere evaluation, it reports MSE 0.0026, PSNR 26.60, SSIM 0.76, and LPIPS 0.10. The paper emphasizes that the method also reconstructs and renders full 360° heads, including the back, without facial tracking.

These two systems occupy different points in the design space. Vid2Avatar-Pro relies on a cross-identity universal prior and canonical front/back maps; GAN-Avatar relies on a personalized 3D-aware GAN and tracker-free appearance learning. Both, however, treat missing back supervision as a first-order modeling problem rather than a minor postprocessing artifact.

5. Editing back-ends and trust back-ends

AvatarBack also appears as a back-end abstraction for operations beyond reconstruction. GeneAvatar targets a different bottleneck: heterogeneous 3DMM-driven volumetric head avatars lack a unified editing interface that can lift a single edited 2D image to a consistent 3D modification across viewpoints and expressions. GeneAvatar provides an expression-aware 3D modification field that is independent of the underlying volumetric field and can therefore function as a universal avatar-editing back-end (Bao et al., 2024).

The model takes a volumetric head avatar $E_R$2, a single edited image $E_R$3, and the camera parameters used to render the original view. It outputs a 3D modification field $E_R$4 composed of a geometry warp and a texture modification. Geometry is produced by $E_R$5, which maps sample points $E_R$6 in edited space to $E_R$7 in original avatar space; texture is produced by $E_R$8, which outputs a per-point modification color $E_R$9 and blending weight $z$0. Both generators are conditioned on FLAME vertices $z$1. Rendering uses standard volumetric accumulation, and single-view fitting is performed by auto-decoding in $z$2-space with

$z$3

where $z$4, $z$5, and $z$6.

GeneAvatar’s training data are distilled rather than manually edited. Geometry supervision comes from Next3D by altering 3DMM shape coefficients, and texture supervision comes from segmentation-based editing or makeup transfer, then PTI inversion. The architecture uses a StyleGAN-like mapping from $z$7 to $z$8, a synthesis network that produces a 256×256×32 feature map, and four planes—one frontal, two side, and one top-of-head. Reported RTX 4090 runtimes are approximately 75 s for geometry editing and approximately 164 s for texture editing, with GPU memory approximately 9.1 GB and model size approximately 234 MB. Limitations include adding new objects such as hats and significantly modifying hairstyles, where results may be incomplete or blurry.

A separate back-end problem is trust. Avatar fingerprinting formalizes the verification task as determining who is driving the motion rather than whose face is shown. The method learns appearance-invariant motion signatures from facial landmark dynamics and is explicitly described as relevant to a system like AvatarBack that must ensure trustworthy avatar communication and enable auditing or backtracking of expression sources (Prashnani et al., 2023).

The dataset contains 161 identities and more than 242,864 synthetic face reenactments. Features are built from 126 facial landmarks per frame and all pairwise Euclidean distances, yielding 7,875-dimensional frame descriptors normalized by the square root of face bounding-box area in a neutral frame. A temporal CNN maps 7875×$z$9 clips to a 128-dimensional embedding. The objective pulls together clips with the same driver across different target identities and pushes away different drivers and time-shuffled clips. With default clip length $G(x,z)$0, the method reports AUC 0.886 on Face-vid2vid, 0.862 on LIA, and 0.867 on TPS. Removing the time-shuffle push term reduces AUC to 0.851. Baselines adapted from Agarwal et al. and ID-Reveal reach 0.726 and 0.720, respectively.

This usage broadens the meaning of AvatarBack. The back-end is no longer only geometric completion; it is also a control, editing, and governance layer for avatar systems that must remain consistent, attributable, and authorized.

6. Applications, limitations, and open problems

The practical applications of AvatarBack span collocated VR, social VR meetings, telepresence, AR/VR, and entertainment. In RevAvatar, the explicit use cases are meetings and social VR, where the 2D restoration output can be cropped to render eye regions on the outward-facing display or can drive the 3D avatar to convey richer expressions externally. In Gaussian head avatars and monocular human avatars, the principal applications are complete, photorealistic, and drivable 3D heads suitable for AR/VR, telepresence, and entertainment. GeneAvatar extends these systems toward high-level editing without changing their internal architectures, while avatar fingerprinting adds authorization and auditing capabilities (Dash et al., 24 May 2025, Xin et al., 28 Aug 2025, Guo et al., 3 Mar 2025, Bao et al., 2024, Prashnani et al., 2023).

Operational telepresence systems show why these functions matter. NimbRo Avatar features a telepresence screen displaying a synthesized image of the operator with facial animation, and the later ANA Avatar XPRIZE system used photorealistic face animation on the avatar head display together with 6D head motion, stereo VR streaming, and spherical rendering. In the finals, NimbRo achieved a perfect 15/15 on both days, with completion times 8:15 and 5:50, and judges with only brief training completed a timed multi-task course (Schwarz et al., 2021, Lenz et al., 2023). This suggests that outward facial display, nonverbal signaling, and embodiment quality are not peripheral concerns; they are central to remote social and manipulation performance.

The limitations are correspondingly broad. In reverse pass-through VR, challenging scenarios include severe occlusions of the upper face, fast head motions causing motion blur, dramatic lighting changes in IR eye cameras, hardware diversity, and the need for device-specific calibration. In back-head completion for Gaussian avatars, failure cases include extremely complex or dynamic hairstyles, severe occlusions, unusual back-head accessories, and lighting mismatches between synthetic and real data. In Vid2Avatar-Pro, sparse or occluded back observations, loose outfits, extreme motions, and extreme lighting remain difficult. In GeneAvatar, adding new objects, hair modifications, extreme expressions, and back-of-head edits remain harder than frontal facial regions. In avatar fingerprinting, low-expressivity drivers and landmark robustness can reduce separability. In immersive telepresence, force-feedback teleoperation becomes unstable beyond approximately 50 ms one-way delay, and locomotion situational awareness benefits from additional side or rear sensing (Dash et al., 24 May 2025, Xin et al., 28 Aug 2025, Guo et al., 3 Mar 2025, Bao et al., 2024, Prashnani et al., 2023, Schwarz et al., 2021, Lenz et al., 2023).

Ethics and safety are equally intrinsic. RevAvatar explicitly notes risks of spoofing, impersonation, and outward displays that may mislead bystanders if identity or expression drift occurs. The stated mitigations are explicit user consent for DP capture and AvatarBack activation, opt-out per session, on-device inference, watermark or indicator when reconstruction is active, secure storage of DP images, differential privacy or encryption where applicable, performance validation across skin tones and different HMDs, and fallback modes when confidence drops. Avatar fingerprinting adds a complementary safeguard by verifying the driver identity independently of the rendered facial appearance. A plausible implication is that future AvatarBack systems will need to combine reconstruction quality, controllability, calibration, and authorization rather than treating them as separable subsystems.

Taken together, the literature frames AvatarBack as a technically heterogeneous but conceptually coherent agenda: restoring the missing parts of avatar presence—face, gaze, back-head, back-side body appearance, editable 3D consistency, and accountable motion provenance—under the severe observability and deployment constraints of real VR, telepresence, and synthetic-media systems.