HairGPT: Strand-as-Language Autoregressive Modeling for Realistic 3D Hairstyle Synthesis

Abstract: Hair is a rich medium of visual and cultural expression, yet its digital modeling remains challenging due to the duality of fluidity and structure. Many existing generative approaches rely primarily on continuous diffusion fields, which entangle global topology with local texture and obscure the semantic and structural organization of hairstyles. To address this, we propose HairGPT, a strand-centric framework that treats strands as generative primitives and formulates realistic 3D hairstyle synthesis as a dual-decoupled autoregressive sequence modeling problem. Our method applies spatial decoupling across semantic scalp regions and structural decoupling along a hierarchical strand representation, progressing from global layout to fine-grained style. We further introduce a geometric tokenizer and region-aware semantic annotations to guide strand-level generation, enabling compositional editing, synthesis of rare and complex hairstyles, and adaptation to stylized domains. By aligning generative modeling with the workflow of digital grooming, HairGPT turns hair generation from opaque texture synthesis into a structured and semantically controllable authoring process, supporting robust semantic conditioning and high-fidelity results across realistic and stylized domains. Project Page: https://haiminluo.github.io/hairgpt/

• The paper presents the main contribution of reframing 3D hair modeling as a strand-level autoregressive language task, enabling explicit structural control.
• It introduces a dual-axis decoupling that separates scalp-level spatial context from strand-level geometry and style, using tokenized quantization for precise modeling.
• Empirical results demonstrate superior geometric fidelity and content alignment over diffusion models, validating its potential for controllable 3D hair synthesis.

HairGPT: Strand-as-Language Autoregressive Modeling for Realistic 3D Hairstyle Synthesis

Problem Formulation and Motivation

Realistic 3D hair modeling remains technically challenging due to the fine-grained balance between global structure and high-frequency local detail. Prior generative frameworks—especially those relying on continuous diffusion or neural fields—entangle topology and texture via holistic latent field synthesis, resulting in limited semantic control and inflexible editing pipelines. These methods struggle to disentangle the anatomical segmentation of scalp regions or represent the explicit organization imposed by artistic, cultural, or identity-driven hairstyle distinctions. This misalignment with human conceptualizations of hair structure impedes controllable and expressive authoring.

The key insight underpinning HairGPT is the reframing of hairstyling as a structured sequence modeling task, treating individual strands as atomic generative units. This enables translating multimodal (image/text) intent into explicit strand assemblies via a dual-decoupled autoregressive process that enforces both spatial and structural modularity and supports robust region-level semantic conditioning.

Dual-Decoupled Representation and Strand Abstraction

HairGPT advances a dual-axis decoupling—scalp-level spatial and strand-level structural hierarchy. At the macroscopic level, the scalp is partitioned into eight semantically meaningful zones (Front, Top, Crown, Nape, bilateral Sides, and Temples) following the Hairmony taxonomy, with a continuous scalp-space density map $D$ providing a global prior for strand root distribution.

At the strand level, geometry is factorized into $(u, c, r)$: root location $u$; low-frequency coarse backbone $c$ (global flow/topology); and high-frequency style residual $r$ (local detail). Guide strands are extracted from raw meshes using DCT-based frequency-aware clustering to obtain a manageable, representative set suitable for sequence modeling. Coarse geometry leverages truncated DCT descriptors, while style residuals are computed in a local frame for invariance to global transformations, yielding a bijective, scale/rotation-invariant parameterization.

Discrete Strand Tokenization and Data Pipeline

The structured representation is discretized into tokens amenable to autoregressive modeling. Strand roots are quantized on a $256 \times 256$ scalp UV grid. A multi-head VQ-VAE product quantizer is employed for both coarse backbone and style residuals, partitioning geometric latents into 4 subspaces and encoding each via a dedicated sub-codebook (coarse: 8192 entries, style: 2048 entries), crucial for both expressivity and codex utilization at practical sequence lengths ($\sim$8 tokens/strand).

Global density maps are compressed via a 2D VQ-VAE into 1024 tokens, supporting density-conditioned autoregressive reasoning. The data pipeline aligns image, text, and geometry triplets through generative augmentation (Qwen-image for visual diversity) and a LoRA-adapted Qwen2.5-VL-7B-Instruct for hierarchical, region-aware semantic annotation, extended with natural language normalization.

The training dataset (120k triplets) amalgamates synthetic priors (DiffLocks, Perm), artist designs, and reconstructions annotated according to the rigorous Hairmony taxonomy, enabling semi-automated yet semantically granular supervision of both global and localized hairstyle attributes.

HairGPT Framework: Multimodal Autoregressive Generation

HairGPT instantiates a decoder-only transformer (Qwen backbone) that models hairstyle synthesis as multimodal conditioned, region-aware, phased autoregressive token prediction. The architecture accepts unified vocabularies spanning hair geometry, density, image, and text, with specialized projectors (frozen DINOv3 for vision, aligner for text) maintaining compatibility and leveraging pretrained semantics.

Generation proceeds hierarchically through three functional phases:

• Density Phase: Predicts the scalp density map, establishing the coarse global structure.
• Layout Phase: Sequentially generates strand root locations for each region.
• Coarse/Style Phase: Refines each strand with low-frequency backbone and high-frequency style residuals, conditioned on fixed roots and context.

Explicit separator tokens ($s_1$, $s_2$, $s_3$) steer the model into mode-specific behavior, enforced through staged training and redundancy masking to prevent autoregressive drift and long-range dependency attenuation.

Empirical Results and Ablations

HairGPT outperforms state-of-the-art diffusion models (DiffLocks) and strand-based methods (HairStep, HAAR) across both image-guided and text-guided synthesis scenarios. It delivers structurally coherent, detail-preserving results, including plausible synthesis of rare and topologically complex categories (e.g., afros, buns, ponytails, diagonal bangs).

Key quantitative metrics on DiffLocks’ evaluation set highlight the model’s strong geometric fidelity and content alignment:

Method	Chamfer Distance (×10⁻³)	DCT-FID (×10⁻³)	Precision	Recall	F-score	CLIP Score
Ours	6.31	8.80	0.824	0.833	0.828	0.592
DiffLocks	7.83	9.97	0.839	0.809	0.824	0.577
HairStep	-	-	-	-	-	0.537

Notably, the dual-decoupled representation (without which the model cannot recover fine details or characteristic curls), multi-head tokenizer, and phased sequence construction are all empirically ablated, confirming their necessity. Alternative approaches (e.g., interleaving geometric tokens, losing staged training) suffer geometric collapse or severe loss of detail.

Applications and Limitations

The explicit, modular representation enables a spectrum of advanced capabilities:

• Cross-domain adaptation (e.g., stylized anime hair via few-shot fine-tuning),
• Semantic and regional editing (e.g., modifying density, local style, or topology via image/text),
• Unified photorealistic avatar generation when integrated with complementary 3D face synthesis models.

However, some trade-offs arise. The eight-token-per-strand encoding, though critical for tractability, constrains local fidelity, potentially smoothing high-curvature or irregular regions. Reliance on VLM-generated annotations, while scalable, does not fully supplant expert segmentation, especially for subtle attributes. Real-time dense hair inference is relegated to interpolation postprocessing, and the sequential autoregressive inference introduces latency compared to parallel diffusion alternatives.

Theoretical and Practical Implications

HairGPT operationalizes a new "strand-as-language" paradigm, aligning generative modeling with the structured logic of digital grooming. By explicitly modeling compositional, region-anchored, and hierarchically modular primitives, this framework provides a foundation for controllable, intent-driven 3D content creation—a crucial prerequisite for downstream tasks such as animation, simulation, interactive authoring, and cognitive alignment with artist workflows.

The approach generalizes beyond static synthesis, pointing toward agentic, self-refining hair generation (e.g., iterative planning, editing, verification across regions and abstraction levels) and robust cross-domain adaptation (art, stylization, avatarics). Hybrid, more expressive local encodings or more efficient parallelized token generation are avenues for further exploration.

Conclusion

HairGPT introduces a semantically aligned, dual-decoupled autoregressive model for realistic strand-based 3D hairstyle synthesis. By reformulating hair as structured language and leveraging advanced multimodal alignment and geometric tokenization, the framework attains strong structural fidelity, fine local detail, and robust region-aware conditioning. The methodology’s flexibility substantiates new possibilities in intuitive, artist-aligned, and agent-driven 3D content generation (2605.08824).