BFSM: 3D Bidirectional Face-Skull Morphable Model (2509.24577v1)

Abstract: Building a joint face-skull morphable model holds great potential for applications such as remote diagnostics, surgical planning, medical education, and physically based facial simulation. However, realizing this vision is constrained by the scarcity of paired face-skull data, insufficient registration accuracy, and limited exploration of reconstruction and clinical applications. Moreover, individuals with craniofacial deformities are often overlooked, resulting in underrepresentation and limited inclusivity. To address these challenges, we first construct a dataset comprising over 200 samples, including both normal cases and rare craniofacial conditions. Each case contains a CT-based skull, a CT-based face, and a high-fidelity textured face scan. Secondly, we propose a novel dense ray matching registration method that ensures topological consistency across face, skull, and their tissue correspondences. Based on this, we introduce the 3D Bidirectional Face-Skull Morphable Model (BFSM), which enables shape inference between the face and skull through a shared coefficient space, while also modeling tissue thickness variation to support one-to-many facial reconstructions from the same skull, reflecting individual changes such as fat over time. Finally, we demonstrate the potential of BFSM in medical applications, including 3D face-skull reconstruction from a single image and surgical planning prediction. Extensive experiments confirm the robustness and accuracy of our method. BFSM is available at https://github.com/wang-zidu/BFSM

• The paper presents a novel bidirectional morphable model that infers shape between the face and skull using a shared latent space, overcoming paired data scarcity.
• It introduces a dense ray matching registration method that ensures topology-consistent alignment between CT scans and high-fidelity face scans for robust reconstructions.
• Experimental results validate BFSM's potential in surgical planning and diagnostics, achieving high accuracy in reconstructing both normal and deformed cases.

BFSM: 3D Bidirectional Face-Skull Morphable Model

Introduction

The paper "BFSM: 3D Bidirectional Face-Skull Morphable Model" presents a cutting-edge bidirectional morphable model that facilitates shape inference between the face and skull using a shared coefficient space. This model addresses critical challenges such as scarcity of paired face-skull data and the need for topological consistency in modeling, thereby enabling applications in remote diagnostics, surgical planning, and medical education. The novelty lies in the data acquisition, modeling, and reconstruction methodologies, which offer a comprehensive framework for face-skull analyses.

Figure 1: We develop bidirectional face-skull morphable models in (a) data, (b) modeling, and (c) reconstruction.

Data Acquisition and Modeling Challenges

A significant barrier to the development of morphable models integrating facial geometry and internal skull anatomy is the limited availability of paired datasets. To address this, the authors created a dataset encompassing over 200 samples, featuring CT-based skulls, faces, and high-fidelity textured face scans. This dataset includes a diverse range of conditions, including normal cases and various craniofacial deformities. The BFSM leverages a novel dense ray matching registration method that conforms to the anatomical topology of face-skull pairs, ensuring consistent geometric alignment across varied cases (Figure 2).

Figure 2: (a) and (b): Overview of our face and skull data acquisition process. (c): Clinician-guided and automated alignment of CT and scanning data.

Dense Ray Matching Registration

The paper introduces a dense ray matching technique specifically for non-rigid registration, crucial for aligning traditional CT scans with high-fidelity face models. This method ensures that anatomical agreements and topological consistency are maintained. Dense ray matching allows each ray to pair a face mesh vertex with a corresponding skull site, facilitating tissue correlation modeling. This approach significantly enhances the robustness of registration for both normal and deformed cases (Figure 3).

Figure 3: Overview of our topology-consistent deformation pipeline. (a) Dense ray matching for face mesh deformation. (b) Dense ray matching for skull mesh deformation.

Bidirectional Face-Skull Morphable Model (BFSM)

The BFSM introduces the Face-Skull Morphable Model (FSMM) and the Tissue Morphable Model (TMM). The FSMM supports bidirectional shape inference between the face and skull using a shared latent space, encapsulating identity, albedo, and geometric transforms. The TMM captures tissue thickness variations, effectively supporting one-to-many facial reconstructions from a given skull, considering individual variability in physiology. This offers a comprehensive platform for numerous applications, from surgical planning to virtual reality.

Figure 4: Results visualization of dense ray matching for topology registration (a), FSMM (b), and TMM (c).

Applications and Experiments

The BFSM exhibits significant potential in healthcare and digital accessibility applications, facilitating accurate reconstructions from single images and predicting postoperative results with high fidelity. In surgical planning, BFSM demonstrates improved outcome predictions, enhancing clinical decision-making and patient communication (Figure 5). Extensive experiments showcase the model's superiority in robustness and accuracy over existing methods, with remarkable predictions in surgical planning showing low error margins.

Figure 5: (a) Reconstruction via FSMM. (b) Facial variation by perturbing TMM coefficients for a given skull.

Conclusion

The BFSM sets a new standard in bidirectional face-skull modeling by overcoming traditional data and modeling constraints. Its contribution to inclusive datasets and innovative modeling methods significantly advances the potential applications in medical simulations and procedural training, promising further advancements in AI-driven healthcare solutions.

Figure 6: Face-skull reconstruction from a single image.

The implications of this research highlight a broader acceptance of morphable models in clinical diagnostics and procedural rehearsals, emphasizing the need for ongoing research in expanding datasets and refining modeling accuracies for diverse and pathological conditions. Future work will likely focus on enhancing model scalability and integration into routine medical imaging practices.