MeshLAM: Feed-Forward One-Shot Animatable Textured Mesh Avatar Reconstruction

Abstract: We introduce MeshLAM, a feed-forward framework for one-shot animatable mesh head reconstruction that generates high-fidelity, animatable 3D head avatars from a single image. Unlike previous work that relies on time-consuming test-time optimization or extensive multi-view data, our method produces complete mesh representations with inherent animatability from a single image in a single forward pass. Our approach employs a dual shape and texture map architecture that simultaneously processes mesh vertices and texture map with extracted image features from a shared transformer backbone, allowing for coherent shape carving and appearance modeling. To prevent mesh collapse and ensure topological integrity during feed-forward deformation, we propose an iterative GRU-based decoding mechanism with progressive geometry deformation and texture refinement, coupled with a novel reprojection-based texture guidance mechanism that anchors appearance learning to the input image. Extensive experiments demonstrate that our method outperforms state-of-the-art approaches in reconstruction quality, animation capability, and computational efficiency. Project page at https://meshlam.github.io.

• The paper introduces a dual-branch architecture that decouples geometric deformation and texture synthesis, enabling efficient one-shot avatar reconstruction.
• It leverages iterative GRU decoders with topology correction and UV texture reprojection to preserve high-frequency details and ensure mesh integrity.
• Quantitative results demonstrate superior performance in PSNR, SSIM, and LPIPS with real-time speeds, making it applicable for VR, telepresence, and content creation.

MeshLAM: Feed-Forward One-Shot Animatable Textured Mesh Avatar Reconstruction

Introduction and Motivation

Advancing the synthesis of animatable, high-fidelity 3D head avatars from single images necessitates overcoming canonical challenges in 3D-aware generation—balancing geometry fidelity, texture detail, animatability, and efficiency. Mesh representations are topologically efficient and support direct deformation for animation, while recent Gaussian-based frameworks achieve visually pleasing results but are inefficient for storing high-frequency appearance due to the explosion in primitive count. MeshLAM presents a feed-forward architecture explicitly targeting the generation of shape-consistent, animatable, textured head avatars from a single image without test-time optimization or multi-view data, leveraging a dual-branch mesh and texture pipeline with iterative refinement, topology correction, and texture reprojection guidance.

Figure 1: The dual-branch framework reconstructs mesh geometry and UV texture from a single image using transformer-based features, iterative GRU decoders with topology and reprojection guidance.

Method Overview

The MeshLAM architecture builds on the FLAME template mesh for inherent animatability and topology. A shared ViT-based feature extractor (frozen DINOv2) processes the input image. Features are delivered to two independent, but synergistic branches:

• Shape branch: Predicts vertexwise geometric deformations with cross-attention to image features and iteratively refines mesh structure using a GRU-based decoder stabilized through topology correction.
• Texture branch: Synthesizes a high-resolution UV texture map by decoding learnable token grids, cross-attending to image features, and employing an iterative GRU for progressive, photometrically guided texture enhancement.

Critically, at each iteration, a reprojection module unwraps the input image and the current synthesis error onto the evolving mesh in UV space, directly supervising texture learning and correcting geometric biases in the deformation process.

Core Innovations Compared to Prior Work

MeshLAM's crucial design innovations include:

• Dual-branch explicit mesh-UV parameterization: Decouples geometric deformation from appearance modeling, resolving the inefficiency and detail loss of Gaussian and volumetric techniques operating on unstructured primitives. Empirically, a compact mesh with 8K vertices suffices for topologically consistent shape, unlike prior LAM-style methods that require $>$80K Gaussians for similar detail and still suffer texture blurring.

  Figure 2: High-fidelity head avatar example—MeshLAM recovers texture detail (tattoos, text) where Gaussian-based LAM fails or appears blurry, despite using only 8K mesh vertices.

• Iterative mesh and texture decoding with topology correction: Direct regression leads to catastrophic mesh collapse, especially for hair or headwear. GRU-based iterative decoding, combined with dynamic topology repair (edge subdivision, remeshing, orientation correction), ensures stability and anatomical correctness.

  Figure 3: Single-pass regression produces collapsed meshes and corrupted geometry; iterative GRU decoding recovers stable, deformable, topologically valid head shapes.

• Texture reprojection supervision in UV-space: At each step, re-projected input image regions and texture errors are fused for texture branch updates, optimizing photorealistic appearance and mitigating drift even for occlusions or ambiguous input.

  Figure 4: Without reprojection guidance, texture becomes blurry (middle); full MeshLAM preserves high-frequency appearance detail via recurrent visual feedback.

• Part-aware semantic deformation: Mesh deformation is anatomically constrained with adaptive per-region displacement bounds and semantic parsing mask consistency, promoting realism of facial components and structurally complex hair/accessories.

  Figure 5: Absence of semantic part constraints leads to implausible facial artifacts; part-aware mesh supervision yields correct structure for hair, face, and accessories.

Quantitative and Qualitative Results

MeshLAM achieves significant improvements over mesh-based and Gaussian-based one-shot avatar methods. On the VFHQ test set, MeshLAM with UNet renderer records PSNR 25.23, SSIM 0.879, and LPIPS 0.061, outperforming ROME and LAM+FLAME. Its mesh-only variant (no UNet) still surpasses mesh-based baselines, with identity cosine similarity (CSIM) 0.948. The pipeline runs at 0.7s/image (5K vertices, 4K tokens), compared to 1.4–5.8s (20K–80K points) for Gaussian LAM with inferior high-frequency recovery.

Generalization and Applications

The explicit mesh/UV parameterization and photometric link enable strong cross-domain adaptation:

• Text-to-3D avatar generation: Leveraging text-to-image diffusion (e.g., SANA, Qwen-Img), MeshLAM produces semantically aligned, animatable avatars from synthetic images.
• Style transfer and editing: Image-based artistic/style editing is realized by first editing the source image and reconstructing the 3D mesh-texture avatar, not requiring retraining or style-specific supervision.

  Figure 6: The framework supports text-driven 3D avatar generation via synthetic image conditioning, and efficient style transfer by combining 2D editing with the mesh/textured pipeline.

Ablation and Stability

Ablations show in detail that both the iterative GRU mesh-texture decoders and the explicit UV texture maps are required for sharp, artifact-free avatars. Per-vertex color or single-pass architectures collapse fidelity (PSNR drops from 25.23 to 18.09 w/o texture map); direct, non-iterative deformation collapses mesh topology; removing semantic/part constraints yields anatomical artifacts, and ablation of the reprojection module immediately induces texture blurring.

Geometric and Texture Visualization

Figure 7: MeshLAM produces detailed, animatable 3D geometry and high-res unwrapped UV textures, with consistent detail under arbitrary expressions.

Robustness Under Realistic and Adverse Inputs

MeshLAM robustly reconstructs under occlusions, challenging lighting, and low-image quality.

Figure 8: Challenging real-world conditions (occlusions, variable lighting) are handled stably by the framework.

Comparative Evaluation

Figure 9: MeshLAM outperforms Portrait4Dv2 and GAGAvatar baselines on detailed texture and mesh accuracy.

Practical and Theoretical Implications

The explicit mesh and UV-texture parameterization is a decisive advance for efficient, feed-forward, high-fidelity 3D avatar construction in low-data regimes. The disentangled geometry/texture modeling resolves many limitations of current 3DMM, NeRF, SDF, and Gaussian-based pipelines, especially for applications requiring real-time, cross-domain, or semantically editable avatars (VR, telepresence, content creation, virtual agents).

The iterative, error-corrective feedback loop closely approximates EM-style or recurrent attention processes, suggesting future generalizations for dynamic head avatars (wrinkle/tongue/soft-tissue), physically-based relighting, or multi-actor interaction. Challenges remain with FLAME template limitations, and future work will benefit from extending morphable models, improving semantic segmentation, and refining direct audio-driven animation.

Conclusion

MeshLAM introduces a dual-branch, feed-forward 3D mesh and UV-texture avatar construction framework with iterative mesh-texture refinement, topology correction, and reprojection supervision, achieving state-of-the-art detail, animatability, and computational efficiency for one-shot avatar generation from a single image (2604.22865). The explicit decomposition and convergence guarantees empirically validated in the paper establish a strong baseline and foundation for generalizable, photo-realistic, real-time digital human synthesis.