HumanNOVA: Photorealistic, Universal and Rapid 3D Human Avatar Modeling from a Single Image

Abstract: In this paper, we present HumanNOVA, a photorealistic, universal, and rapid model for generating 3D human avatars from a single RGB image. Achieving both photorealism and generalization is challenging due to the scarcity of diverse, high-quality 3D human data. To address this, we build a scalable data generation pipeline that follows two strategies. The first one is to leverage existing rigged assets and animate them with extensive poses from daily life. The second strategy is to utilize existing multi-camera captures of humans and employ fitting to generate more diverse views for training. These two strategies enable us to scale up to 100k assets, significantly enhancing both the quantity and the diversity of data for robust model training. In terms of the architecture, HumanNOVA adopts a feed-forward, token-conditioned avatar modeling framework that allows fast inference in less than one second and requires no test-time optimization. Given an input image and an estimated simplified human mesh (SMPL) without detailed geometry or appearance, the model first encodes both inputs into compact token representations. These tokens then act as conditioning signals and are fused through cross-attention to construct a triplane-based 3D avatar representation. Extensive experiments on multiple benchmarks demonstrate the superiority of our approach, both quantitatively and qualitatively, as well as its robustness under diverse input image conditions. Project page at https://HumanNOVA.github.io .

• The paper introduces a feed-forward transformer-based framework that rapidly reconstructs high-fidelity 3D human avatars from a single image.
• It integrates SMPL priors with multi-modal encoding and token-conditioned mapping to achieve precise pose and texture inference.
• The approach leverages a large-scale composite dataset and high-resolution triplane rendering to outperform state-of-the-art methods.

HumanNOVA: Photorealistic, Universal, and Rapid 3D Human Avatar Modeling from a Single Image

Introduction and Motivation

HumanNOVA addresses single-image 3D human avatar reconstruction with an emphasis on photorealism, generalization across diverse scenarios, and high-speed inference. The framework targets critical application domains including VR, telepresence, and HCI, where avatar fidelity and computational efficiency are essential. Existing methods leveraging neural volumetric representations (e.g., NeRF, 3DGS) combined with strong structural priors (e.g., SMPL/SMPL-X parametric models) produce high-quality outcomes but commonly incur excessive computational overhead due to reliance on instance-level optimization or iterative generative refinement, such as diffusion-based view completion. HumanNOVA aims to bridge this efficiency-fidelity gap by introducing a feed-forward, large-model paradigm, specifically tailored for the human domain, that integrates domain-specific priors while leveraging large-scale multi-source training data.

Model Architecture

HumanNOVA adopts a transformer-centric architecture inspired by LRM-based feed-forward models for scalable single-image-to-3D reconstruction. The pipeline consists of three principal modules:

1. Multi-Modal Encoding: The input RGB image is tokenized using a DINOv2 encoder, producing visual tokens. Concurrently, an SMPL-estimated simplified human mesh is processed using Point Transformer v3 to produce mesh tokens. This leverages the SMPL prior for structural regularization, providing robust pose and pose-consistent surface cues even for in-the-wild images.
2. Token-Conditioned Mapping: The core mapping network, following PointInfinity and SF3D variants, fuses visual and mesh tokens via multi-headed cross attention. The fusion tokens are progressively refined through multiple attention layers to produce a structured 3D triplane latent representation.
3. Triplane-based Rendering: The fused latent tokens define three orthogonal projection planes (triplanes), which are decoded via a NeRF-style MLP and rendered into 2D images under novel camera viewpoints by standard ray-marching. Supervision is enforced on both RGB patches and geometric masks for each synthesized view.

   Figure 1: The HumanNOVA network first estimates a simplified human mesh, encodes image and mesh inputs, and maps their fused features into a high-resolution 3D triplane representation for fast photorealistic rendering.

Data Generation Strategy

A core contribution of HumanNOVA is its large-scale composite training dataset, addressing the chronic scarcity and domain-bias of existing human scans. The data generation pipeline comprises two pipelines:

• Synthetic Asset Animation: 78k high-quality rigged assets from SynBody are animated with a broad spectrum of daily-life poses sampled from the AMASS dataset. Each asset undergoes multi-view rendering from stochastically sampled azimuth and elevation angles, producing high diversity in pose, shape, and clothing.
• Real-World Multiview Fitting: 22k physically captured samples (THuman2, CustomHuman, MVHumanNet, DNA-Rendering) are fit using 3DGS optimized from SMPL-X mesh initializations. The resulting optimized 3D Gaussians facilitate photorealistic re-rendering in novel canonical viewpoints, aligning synthetic data distribution with real-world performance.

The final dataset includes 100k distinct assets with ∼2.6M images, representing a 20-fold increase over prior works.

Figure 3: Sampled rows from the training set: real-world fitted multi-view captures (top) and synthetic animated assets (bottom) provide strong coverage across appearance and pose.

Training Protocol and Objectives

The HumanNOVA model is supervised with multi-faceted objectives, including patch-wise RGB loss, mask consistency loss for silhouette alignment, and LPIPS for perceptual appearance preservation. All objectives are computed via importance-weighted patch sampling to prioritize human foreground content and accelerate learning of fine-scale details.

Key training parameters:

• Triplane resolution: 96 (substantially increased over prior LRM baselines for human-level detail)
• Use of 4 rendered supervision views per instance per iteration
• AdamW optimizer, batch size 64, learning rate 6 × 10⁻⁴

Experimental Results

Quantitative Comparisons

On CustomHuman, THuman2, and 2K2K benchmarks, HumanNOVA surpasses all prior state-of-the-art models (Real3D, SF3D, Hunyuan2, PaMIR, SiFU, SiTH), achieving:

• LPIPS improvement: Over 40% relative gain compared to top baselines (e.g., SiTH) across all benchmarks and both frontal and side-view inputs
• F-Score gain: Exceeds 90% relative improvement in geometric F-Score, highlighting marked advances in mesh fidelity and surface correspondence
• Ablation studies: Removing SMPL prior or using a low-resolution triplane both significantly degrade output quality, emphasizing the dual necessity of explicit structural priors and high-resolution 3D representations

Qualitative Results

Figure 4: HumanNOVA achieves superior structure and texture reproduction versus previous methods, displaying both benchmark and in-the-wild (bottom) reconstructions.

Figure 5: Robustness demonstrated on diverse in-the-wild imagery; notable failure cases include poor texture inference on occluded/complex clothing.

Failures primarily manifest as texture ambiguity on occluded regions or clothing with rare topology (e.g., dresses); this is inherently limited by single-view observability and training data bias.

Ablation and Architectural Analysis

Figure 6: Enabling the SMPL prior dramatically improves anatomical plausibility and garment fit over LRM-only baselines.

Figure 7: HumanNOVA exhibits resilience to moderate SMPL estimation errors due to fusion with strong image cues.

Figure 2: Model performance drops when removing generated data modalities, underscoring the importance of multi-source training.

Figure 8: Excluding mesh priors or reducing triplane resolution yields visible decline in mesh structure and texture detail.

Implications and Future Directions

HumanNOVA demonstrates that the synergy of large-scale, compositionally rich training data and hybrid transformer architectures can overcome the classic speed–quality trade-off in 3D human avatar creation from a single image. The adoption of token-based conditional fusion, triplane-based intermediate representations, and incorporation of parametric mesh priors set new baselines for rapid, generalizable human digitization.

From a practical perspective, HumanNOVA immediately enables scalable avatar creation pipelines for AR/VR, telepresence, and HCI with sub-second latency and broad generalization to in-the-wild scenarios. Theoretically, the results suggest that category-specialized LRM design, when paired with domain-aware priors (e.g., SMPL/SMPL-X encoding), is a critical step for achieving robust performance where general-category LRMs saturate.

Open research challenges identified include robust reasoning about human-object interactions, addressing severe occlusions, and improving the inference of unobserved regions (e.g., back-of-body textures) using even stronger conditional priors or learned view hallucination modules. The data generation methodologies outlined also indicate a promising direction for building benchmark-scale datasets in other 3D reconstruction domains.

Conclusion

HumanNOVA advances the state-of-the-art in single-image 3D human avatar reconstruction by integrating high-resolution triplane architectures, token-based multi-modal fusion with mesh priors, and massively augmented training corpora covering synthetic and real-world variants. The framework achieves competitive photorealism, structural accuracy, and near real-time inference. Further progress in input robustness, occlusion reasoning, and human–object interaction modeling is anticipated as future work in this trajectory.

Reference: "HumanNOVA: Photorealistic, Universal and Rapid 3D Human Avatar Modeling from a Single Image" (2606.02573).