3D Bidirectional Face-Skull Morphable Model

• The paper introduces BFSM as a joint statistical model combining facial and skull geometry with a shared identity space, enabling bidirectional inference.
• It employs a dual PCA-based framework with FSMM for coupled hard-tissue modeling and TMM for soft-tissue variability, capturing one-to-many skull-to-face ambiguities.
• The model demonstrates robust performance in clinical applications like surgical planning and forensic reconstruction, validated with precise NRMSE, Recall, and Chamfer metrics.

Searching arXiv for the cited BFSM-related papers to ground the article. The 3D Bidirectional Face-Skull Morphable Model (BFSM) is a joint statistical model of external facial surface, internal skull geometry, and their dense tissue relationship. In the formulation introduced as BFSM, bidirectionality denotes that the same shared coefficient space can support both skull-to-face inference and face-to-skull inference, while a separate tissue model captures the one-to-many ambiguity inherent in craniofacial reconstruction by allowing multiple plausible facial soft-tissue realizations for a fixed skull (Wang et al., 29 Sep 2025). The model is designed for settings in which face-only 3D morphable models are insufficient, including remote diagnostics, surgical planning, medical education, physically based facial simulation, forensic reconstruction, and monocular recovery of paired face-skull anatomy.

1. Position within face-skull modeling

BFSM should be distinguished from earlier anatomically aware models that couple face and skull information without instantiating a fully bidirectional joint morphable model. The immediate predecessor most closely aligned with BFSM goals is SCULPTOR, which jointly models mandible, maxilla, outer facial geometry, and appearance through shared geometry latents and a unified template; however, it is not described as an explicitly invertible or probabilistic bidirectional model with formal skull-to-face and face-to-skull conditional inference (Qiu et al., 2022). A different line is represented by Skull-to-Face, which formulates reconstruction as anatomy-guided skull-to-face synthesis using sparse tissue-depth statistics, a diffusion-based facial prior, and DECA/FLAME latent optimization; it does not learn a shared skull-face generative space and is operationally one-way (Liang et al., 2024). BFSM advances beyond both by combining a shared face-skull coefficient space with a separate dense tissue model, thereby making bidirectionality and one-to-many skull-conditioned facial variation first-class components of the representation (Wang et al., 29 Sep 2025).

Model	Core representation	Relation to BFSM
SCULPTOR	Unified parametric generator of skull, face, and appearance	Strong partial match; practical two-way inference, but not a strict BFSM
Skull-to-Face	Tissue-depth PCA + diffusion initialization + DECA/FLAME fitting	Skull-to-face pipeline, not a joint bidirectional model
BFSM	Joint face-skull PCA plus tissue PCA	Explicit shared-coefficient bidirectional model

This positioning matters because the term “bidirectional” can easily be overstated. In BFSM, it does not imply a deterministic one-skull-one-face mapping. Instead, the model combines a shared identity space for the coupled hard-tissue and soft-tissue geometry with a separate tissue-thickness component that preserves ambiguity rather than eliminating it (Wang et al., 29 Sep 2025).

2. Dataset construction and dense anatomical correspondence

BFSM is built from a dataset of over 200 samples. The model-building subset uses 161 identities and, after horizontal flipping, 322 samples. Each case contains three aligned modalities: a CT-based skull, a CT-based face, and a high-fidelity textured face scan. CT data were acquired with a Philips Brilliance iCT 128-slice spiral CT scanner focused on the craniofacial region, while surface scans were captured with the 3dMD Face System, with coverage from ear to ear horizontally and from hairline to neck vertically at 5 megapixels (Wang et al., 29 Sep 2025).

A defining feature of the dataset is the inclusion of both normative anatomy and craniofacial deformity cases. The listed deformities include cleft lip, Treacher Collins syndrome, hemifacial microsomia, meningoencephalocele, and progressive hemifacial atrophy, and the held-out test set additionally includes cases such as maxillary retrusion, mandibular protrusion, midface depression, and bimaxillary protrusion (Wang et al., 29 Sep 2025). The paper frames this inclusion as an inclusivity issue: models restricted to typical anatomy generalize poorly to deformities and risk reinforcing exclusion in healthcare and anatomy-aware digital modeling.

Preprocessing is clinician-involved. Clinical experts remove anatomically irrelevant internal cranial tissues, scattered noise artifacts, and redundant structures. Because CT-derived face surfaces lack texture and typically show closed eyes, the CT face serves primarily as an intermediate anatomical bridge for aligning the high-quality 3D scan with the CT skull. Clinicians manually annotate anatomical landmarks on the CT face and scanned face, use them for initial rigid alignment, refine alignment with professional medical modeling software such as Materialise 3-matic Medical, and then review and refine the result (Wang et al., 29 Sep 2025). Institutional Review Board approval, written informed consent, and adherence to the Declaration of Helsinki are reported.

The registration pipeline introduces dense ray matching to create topologically consistent correspondences between face, skull, and tissue. A generic deformation module is defined as

$$\mathbf{X}_{\mathrm{r}} = \boldsymbol{\Phi}_\mathrm{D}(\mathbf{X}, \mathbf{X}', P, P')$$

with objective

$$\begin{aligned} \mathcal{L}_{\text{D}} &= \mathcal{L}_{\text{mesh}} + \lambda \mathcal{L}_{\text{smooth}} + \mu \mathcal{L}_{\text{P}} \ &= \sum_{i=1}^{N} \left\| \mathbf{T}(v_i) - \Pi(v_i, \mathbf{X}') \right\|^2 + \lambda \sum_{i=1}^{N} \left\| \Delta \mathbf{T}(v_i) \right\|^2 + \mu \left\| \mathbf{T}(P) - P' \right\|^2 . \end{aligned}$$

The deformation field is optimized as a vertex affine field using AMSGrad in PyTorch (Wang et al., 29 Sep 2025).

For the face, the target 3D scan is first fit with BFM using differentiable rendering under landmark, depth, texture, and semantic part segmentation supervision to obtain a topology-consistent initialization. Dense correspondences are then generated by casting rays along both positive and negative vertex normals from the initialized face mesh to the target scan. For the skull, the already registered face acts as an anatomical scaffold: rays are cast from face vertices toward mapped endpoints in or near the occipital region, and ray-skull intersections are recorded on both template and subject skulls. This produces ordered skull correspondences and dense tissue vectors

$d_{j,k} = v_j - c_{j,k},$

where each vector encodes both magnitude and direction from skull hit point to face vertex (Wang et al., 29 Sep 2025). The registered template resolutions are $N_\mathrm{F}=35709$ face vertices and $N_\mathrm{S}=23208$ skull vertices.

3. FSMM and TMM: the core BFSM formulation

BFSM consists of two PCA-based statistical models built after dense registration: the Face-Skull Morphable Model (FSMM) and the Tissue Morphable Model (TMM). FSMM is the coupled face-skull shape model. It is defined on concatenated face and skull vertices,

$$\begin{aligned} V_\mathrm{model\_1}^{\mathrm{FS}} &= \left[ V_\mathrm{model\_1}^\mathrm{F} \,\middle\|\, V_\mathrm{model\_1}^\mathrm{S} \right] \ &= \boldsymbol{R}(\beta_a)\left(\overline V + \beta_{id}\boldsymbol{B}_{id}\right) + \beta_t, \ A_\mathrm{model\_1}^\mathrm{F} &= \overline A + \beta_{alb}\boldsymbol{B}_{alb}. \end{aligned}$$

Here, $\overline V$ is the mean joint face-skull shape, $\boldsymbol{B}_{id}$ is the PCA basis learned from concatenated face-skull shapes, $\beta_{id}$ is the identity coefficient vector, $\beta_a$ and $$\begin{aligned} \mathcal{L}_{\text{D}} &= \mathcal{L}_{\text{mesh}} + \lambda \mathcal{L}_{\text{smooth}} + \mu \mathcal{L}_{\text{P}} \ &= \sum_{i=1}^{N} \left\| \mathbf{T}(v_i) - \Pi(v_i, \mathbf{X}') \right\|^2 + \lambda \sum_{i=1}^{N} \left\| \Delta \mathbf{T}(v_i) \right\|^2 + \mu \left\| \mathbf{T}(P) - P' \right\|^2 . \end{aligned}$$0 are rotation and translation coefficients, and $$\begin{aligned} \mathcal{L}_{\text{D}} &= \mathcal{L}_{\text{mesh}} + \lambda \mathcal{L}_{\text{smooth}} + \mu \mathcal{L}_{\text{P}} \ &= \sum_{i=1}^{N} \left\| \mathbf{T}(v_i) - \Pi(v_i, \mathbf{X}') \right\|^2 + \lambda \sum_{i=1}^{N} \left\| \Delta \mathbf{T}(v_i) \right\|^2 + \mu \left\| \mathbf{T}(P) - P' \right\|^2 . \end{aligned}$$1 is the facial albedo basis (Wang et al., 29 Sep 2025).

The decisive modeling choice is that face and skull are not assigned separate identity codes. Because PCA is applied to the joint vector $$\begin{aligned} \mathcal{L}_{\text{D}} &= \mathcal{L}_{\text{mesh}} + \lambda \mathcal{L}_{\text{smooth}} + \mu \mathcal{L}_{\text{P}} \ &= \sum_{i=1}^{N} \left\| \mathbf{T}(v_i) - \Pi(v_i, \mathbf{X}') \right\|^2 + \lambda \sum_{i=1}^{N} \left\| \Delta \mathbf{T}(v_i) \right\|^2 + \mu \left\| \mathbf{T}(P) - P' \right\|^2 . \end{aligned}$$2, the same identity coefficient vector $$\begin{aligned} \mathcal{L}_{\text{D}} &= \mathcal{L}_{\text{mesh}} + \lambda \mathcal{L}_{\text{smooth}} + \mu \mathcal{L}_{\text{P}} \ &= \sum_{i=1}^{N} \left\| \mathbf{T}(v_i) - \Pi(v_i, \mathbf{X}') \right\|^2 + \lambda \sum_{i=1}^{N} \left\| \Delta \mathbf{T}(v_i) \right\|^2 + \mu \left\| \mathbf{T}(P) - P' \right\|^2 . \end{aligned}$$3 controls both modalities. This shared coefficient space is the mechanism that makes face-to-skull and skull-to-face inference possible within a single morphable model (Wang et al., 29 Sep 2025).

TMM addresses the fact that skull shape does not uniquely determine facial soft tissue. Rather than representing tissue as sparse scalar depths at anthropometric landmarks, BFSM models dense 3D tissue vectors. The TMM is defined as

$$\begin{aligned} \mathcal{L}_{\text{D}} &= \mathcal{L}_{\text{mesh}} + \lambda \mathcal{L}_{\text{smooth}} + \mu \mathcal{L}_{\text{P}} \ &= \sum_{i=1}^{N} \left\| \mathbf{T}(v_i) - \Pi(v_i, \mathbf{X}') \right\|^2 + \lambda \sum_{i=1}^{N} \left\| \Delta \mathbf{T}(v_i) \right\|^2 + \mu \left\| \mathbf{T}(P) - P' \right\|^2 . \end{aligned}$$4

where $$\begin{aligned} \mathcal{L}_{\text{D}} &= \mathcal{L}_{\text{mesh}} + \lambda \mathcal{L}_{\text{smooth}} + \mu \mathcal{L}_{\text{P}} \ &= \sum_{i=1}^{N} \left\| \mathbf{T}(v_i) - \Pi(v_i, \mathbf{X}') \right\|^2 + \lambda \sum_{i=1}^{N} \left\| \Delta \mathbf{T}(v_i) \right\|^2 + \mu \left\| \mathbf{T}(P) - P' \right\|^2 . \end{aligned}$$5 is the mean tissue vector field, $$\begin{aligned} \mathcal{L}_{\text{D}} &= \mathcal{L}_{\text{mesh}} + \lambda \mathcal{L}_{\text{smooth}} + \mu \mathcal{L}_{\text{P}} \ &= \sum_{i=1}^{N} \left\| \mathbf{T}(v_i) - \Pi(v_i, \mathbf{X}') \right\|^2 + \lambda \sum_{i=1}^{N} \left\| \Delta \mathbf{T}(v_i) \right\|^2 + \mu \left\| \mathbf{T}(P) - P' \right\|^2 . \end{aligned}$$6 is the PCA basis of tissue vectors, $$\begin{aligned} \mathcal{L}_{\text{D}} &= \mathcal{L}_{\text{mesh}} + \lambda \mathcal{L}_{\text{smooth}} + \mu \mathcal{L}_{\text{P}} \ &= \sum_{i=1}^{N} \left\| \mathbf{T}(v_i) - \Pi(v_i, \mathbf{X}') \right\|^2 + \lambda \sum_{i=1}^{N} \left\| \Delta \mathbf{T}(v_i) \right\|^2 + \mu \left\| \mathbf{T}(P) - P' \right\|^2 . \end{aligned}$$7 is the individual-specific tissue coefficient vector, and $$\begin{aligned} \mathcal{L}_{\text{D}} &= \mathcal{L}_{\text{mesh}} + \lambda \mathcal{L}_{\text{smooth}} + \mu \mathcal{L}_{\text{P}} \ &= \sum_{i=1}^{N} \left\| \mathbf{T}(v_i) - \Pi(v_i, \mathbf{X}') \right\|^2 + \lambda \sum_{i=1}^{N} \left\| \Delta \mathbf{T}(v_i) \right\|^2 + \mu \left\| \mathbf{T}(P) - P' \right\|^2 . \end{aligned}$$8 is a global scalar controlling overall tissue thickness scaling (Wang et al., 29 Sep 2025). The induced face-surface point set is then constructed by

$$\begin{aligned} \mathcal{L}_{\text{D}} &= \mathcal{L}_{\text{mesh}} + \lambda \mathcal{L}_{\text{smooth}} + \mu \mathcal{L}_{\text{P}} \ &= \sum_{i=1}^{N} \left\| \mathbf{T}(v_i) - \Pi(v_i, \mathbf{X}') \right\|^2 + \lambda \sum_{i=1}^{N} \left\| \Delta \mathbf{T}(v_i) \right\|^2 + \mu \left\| \mathbf{T}(P) - P' \right\|^2 . \end{aligned}$$9

followed by topology-consistent face recovery,

$d_{j,k} = v_j - c_{j,k},$0

The separation between FSMM and TMM is structurally important. FSMM models anatomically aligned coupled identity geometry; TMM models soft-tissue variability around that geometry. This design allows the skull to remain fixed while the face varies through tissue parameters, thereby encoding one-to-many skull-conditioned reconstruction rather than forcing a unique facial output (Wang et al., 29 Sep 2025).

4. Inference pathways and application domains

BFSM supports several inference modes. In direct 3D terms, fitting the observed face or the observed skull to the corresponding part of FSMM yields a shared identity coefficient estimate, from which the unobserved modality can be read out. The paper conceptually supports both directions, but it does not provide explicit closed-form inference equations for pure 3D-to-3D bidirectional fitting in the provided text (Wang et al., 29 Sep 2025). The practical implication is that bidirectionality is realized through shared-latent reconstruction rather than through an explicitly derived inverse operator.

For image-based inference, BFSM includes a monocular reconstruction system. A backbone $d_{j,k} = v_j - c_{j,k},$1 takes a single $d_{j,k} = v_j - c_{j,k},$2 face image and predicts FSMM coefficients including identity $d_{j,k} = v_j - c_{j,k},$3, pose $d_{j,k} = v_j - c_{j,k},$4, albedo $d_{j,k} = v_j - c_{j,k},$5, and appearance components including spherical-harmonic lighting $d_{j,k} = v_j - c_{j,k},$6 in the rendering pipeline. Because the same identity coefficients determine both face and skull, reconstructing the face also reconstructs the skull. Approximately 150K in-the-wild face images are used for training, excluding expressive faces to better match the neutral FSMM (Wang et al., 29 Sep 2025).

Tissue fitting can then be performed on the reconstructed pair. The paper estimates tissue vectors $d_{j,k} = v_j - c_{j,k},$7 using the same ray-intersection formulation as in registration and fits TMM coefficients through a gradient-based optimization framework,

$d_{j,k} = v_j - c_{j,k},$8

Once fitted, the skull can be held fixed while $d_{j,k} = v_j - c_{j,k},$9 or $N_\mathrm{F}=35709$0 is perturbed to generate multiple plausible facial variations (Wang et al., 29 Sep 2025).

The stated application range is broad: remote diagnostics, surgical planning, medical education, physically based facial simulation, forensic face-from-skull reconstruction, skull-from-face inference, and single-image paired face-skull reconstruction (Wang et al., 29 Sep 2025). The most clinically grounded application is surgical planning prediction. The procedure takes a preoperative face $N_\mathrm{F}=35709$1, a preoperative skull $N_\mathrm{F}=35709$2, and a clinician-designed skull modification plan $N_\mathrm{F}=35709$3; estimates preoperative tissue thickness $N_\mathrm{F}=35709$4; applies the planned skull geometry; and predicts a postoperative face $N_\mathrm{F}=35709$5. The system also visualizes intermediate morphs and displacement heatmaps intended to support surgeon planning, doctor-patient communication, and expectation management (Wang et al., 29 Sep 2025).

5. Empirical performance

The experimental program covers image-based paired face-skull reconstruction, upper-bound representational capability, registration ablation, and clinical demonstration. For image-based paired reconstruction, the reported baselines are SMIRK+FSMM, DECA+FSMM, 3DDFA+FSMM, Deep3D+FSMM, and the proposed monocular BFSM system. Distances are reported in millimeters using NRMSE, Recall, and Chamfer Distance (Wang et al., 29 Sep 2025).

Input setting	Face: NRMSE / Recall / Chamfer	Skull: NRMSE / Recall / Chamfer
Frontal view	2.829 / 0.847 / 5.753	1.933 / 0.935 / 4.283
Side view	2.420 / 0.911 / 4.963	1.726 / 0.958 / 3.911

These are the best reported results in the corresponding comparison table. The paper notes that the gains over Deep3D+FSMM are modest but consistently best overall (Wang et al., 29 Sep 2025).

The upper-bound representation experiment fits each morphable model directly to ground-truth 3D face meshes, thereby isolating representational capacity from inference quality. The strongest reported BFSM configuration, with $N_\mathrm{F}=35709$6 and $N_\mathrm{F}=35709$7, obtains NRMSE $N_\mathrm{F}=35709$8, Recall $N_\mathrm{F}=35709$9, and Chamfer $N_\mathrm{S}=23208$0. By comparison, the listed FLAME and BFM variants underperform on the same test set, which includes deformity cases (Wang et al., 29 Sep 2025). This supports the claim that explicit inclusion of pathological anatomy broadens representational coverage.

The registration ablation varies the number of ray origins $N_\mathrm{S}=23208$1 and endpoints $N_\mathrm{S}=23208$2. The best reported NRMSE, $N_\mathrm{S}=23208$3, is obtained with $N_\mathrm{S}=23208$4 and $N_\mathrm{S}=23208$5, while lower-density settings perform slightly worse (Wang et al., 29 Sep 2025). The paper does not provide a broader quantitative comparison against previous registration methods in the provided text, so its registration advantage is quantified primarily through ablation and qualitative heatmaps.

The surgical planning demonstration compares BFSM with ProPlan CMF™ 3.0 on an illustrated case. The reported mean prediction error against the real postoperative face is $N_\mathrm{S}=23208$6 for BFSM and $N_\mathrm{S}=23208$7 for ProPlan (Wang et al., 29 Sep 2025). This is a clinically suggestive result, though the paper presents it as a proof of potential rather than as large-cohort validation.

6. Limits, misconceptions, and research significance

A recurring misconception is that “bidirectional” implies exact invertibility or uniqueness. BFSM does not claim that a skull uniquely determines a face. On the contrary, the TMM is introduced precisely to model one-to-many facial reconstructions for a fixed skull, including changes such as fat over time and broader soft-tissue variation (Wang et al., 29 Sep 2025). A second misconception is that BFSM is a purely end-to-end learned system. In fact, the overall framework is hybrid: statistical PCA models for face, skull, and tissue; optimization-based deformation registration; learning-based monocular coefficient regression; and differentiable rendering during initialization and image supervision (Wang et al., 29 Sep 2025).

Its limitations are also explicit or directly evident. Although over 200 samples is large for this niche problem, the dataset remains small relative to mainstream face-model corpora. Detailed demographic balance is not reported in the provided text. Pathology coverage, while broader than in previous paired face-skull datasets, is still incomplete. Model construction depends on CT-derived paired data, which are expensive and clinically constrained to acquire. The surgical planning results are promising but not presented as large-scale clinical validation. Runtime and hardware details are not reported. The monocular reconstructor is trained with expressive faces excluded, which may limit robustness on expression-rich images (Wang et al., 29 Sep 2025).

Against this backdrop, BFSM’s research significance is best understood by contrast with the two most relevant prior lines. SCULPTOR already showed that a unified template and shared geometry latents can make anatomically coupled face-skull generation practical, but it stopped short of an explicit shared-coefficient bidirectional morphable model (Qiu et al., 2022). Skull-to-Face showed that tissue statistics can be used to constrain skull-conditioned facial reconstruction, but only through sparse landmark correspondences and a face-side DECA/FLAME optimization loop (Liang et al., 2024). BFSM integrates the joint latent idea and the tissue-variation idea into a single anatomy-aware morphable-model framework with dense, topologically consistent correspondence (Wang et al., 29 Sep 2025).

A plausible implication is that BFSM is especially relevant whenever internal anatomy is not merely a hidden nuisance variable but a required part of the representation. That includes research on paired craniofacial reconstruction, anatomy-aware simulation, and clinically interpretable planning. The same evidence also suggests caution: the model is most compelling as a research and preclinical tool, and its use in real clinical decision-making remains contingent on broader validation, more detailed population reporting, and continued study of the irreducible ambiguity in face-skull relationships (Wang et al., 29 Sep 2025).