3DPain: Synthetic Facial Pain Dataset

• 3DPain is a synthetic dataset that addresses demographic imbalance and enables precise pain assessment with controlled AU activations and clinically calibrated PSPI scores.
• It employs a three-stage pipeline combining 3D mesh generation, diffusion-based texturing, and AU-driven face rigging to synthesize paired neutral and pain images.
• ViTPain utilizes cross-modal distillation with pain-region heatmaps to improve both the accuracy and interpretability of automated facial pain analysis.

3DPain is a large-scale synthetic dataset specifically designed for automated pain assessment. It was introduced together with ViTPain, a Vision Transformer based cross-modal distillation framework, to address two constraints that have limited progress in facial pain analysis: severe demographic and label imbalance in existing datasets and the inability of prior generative models to precisely control facial action units (AUs), facial structure, or clinically validated pain levels. Its three-stage framework generates diverse 3D meshes, textures them with diffusion models, and applies AU-driven face rigging to synthesize multi-view faces with paired neutral and pain images, AU configurations, PSPI scores, and the first dataset-level annotations of pain-region heatmaps (Lin et al., 20 Sep 2025).

1. Concept and problem setting

Automated pain assessment from facial expressions is crucial for non-communicative patients, such as those with dementia (Lin et al., 20 Sep 2025). The motivation for 3DPain is rooted in two limitations described explicitly in the source work: existing datasets exhibit severe demographic and label imbalance due to ethical constraints, and current generative models cannot precisely control facial action units, facial structure, or clinically validated pain levels (Lin et al., 20 Sep 2025).

The dataset is positioned against a clinical-data regime in which rare, high-intensity pain states are difficult or unethical to collect at scale. The 3DPain description contrasts this with existing resources such as UNBC-McMaster, described there as having only 25 predominantly white/young subjects and approximately 89% low-pain frames (Lin et al., 20 Sep 2025). A related line of work on synthetic pain videos likewise frames traditional pain-data collection as ethically and logistically challenging, and reports that synthetic generation can provide an ethical and scalable alternative for video-based pain recognition (Nasimzada et al., 2024).

This framing also intersects with cohort-specific validity concerns. Vision-based pain monitoring has been validated directly on older adults with and without dementia, a population described as underrepresented in existing facial expression datasets of pain (Rezaei et al., 2021). In that sense, 3DPain is best understood as an attempt to expand demographic coverage and label support within a clinically grounded facial-pain representation space rather than as a mere image-synthesis benchmark.

2. Dataset composition and annotation schema

3DPain comprises 82,500 images and 25,000 pain expression heatmaps sampled from 2,500 synthetic identities, with distributions described as balanced across age, gender, and ethnicity (Lin et al., 20 Sep 2025). The detailed demographic summary provided for the dataset includes 646 Latino, 460 White, 469 South Asian, 585 Middle Eastern, 258 East Asian, and 82 Black identities; ages from young adults to elderly; and 1,723 men and 777 women (Lin et al., 20 Sep 2025).

The dataset deliberately samples all pain intensities and action units uniformly, ensuring representation of rare, high-PSPI pain states that are unethical or infeasible to record in real clinical scenarios (Lin et al., 20 Sep 2025). This uniformity is central to its role as a supervision source for automated pain assessment, because it couples demographic diversity with controlled label coverage.

Each sample is richly annotated with continuous AU configurations for six pain-critical AUs, exact PSPI values, pain-region heatmaps, demographic metadata, and paired neutral/pain images (Lin et al., 20 Sep 2025). The pain-critical AUs are AU4, AU6, AU7, AU9, AU10, and AU43, and the PSPI value is defined as

$$\text{PSPI} = \text{AU}_4 + \max(\text{AU}_6, \text{AU}_7) + \max(\text{AU}_9, \text{AU}_{10}) + \text{AU}_{43}.$$

Component	Description	Role
Images	82,500 images	Multi-view facial renders
Heatmaps	25,000 pain expression heatmaps	Spatial localization of pain expression intensity
Identities	2,500 synthetic identities	Demographic coverage
Labels	AU configurations, PSPI, metadata, paired neutral/pain images	Supervised and interpretable learning

The source paper emphasizes that this annotation depth supports supervised, contrastive, and interpretability-focused research (Lin et al., 20 Sep 2025). A plausible implication is that 3DPain is intended not only for end-task classification, but also for mechanistic modeling of AU-level and region-level pain expression.

3. Three-stage synthesis pipeline

The 3DPain generation process is organized as a three-stage pipeline (Lin et al., 20 Sep 2025). In the first stage, 3D mesh creation uses the FLAME parametric 3D face model to generate identity- and demographically-controlled neutral face meshes encoding structure, age, ethnicity, and gender. The depth image from the FLAME mesh is then used as guidance for texture synthesis (Lin et al., 20 Sep 2025).

In the second stage, texturing is performed with diffusion models. Kandinsky 2.2, with ControlNet depth conditioning, generates photorealistic neutral textures from 3D mesh depth maps while maintaining structural realism across views and demographics. Hunyuan3D 2.1 generates physically based rendering textures mapped to FLAME meshes, capturing fine-grained ethnic, age, and skin details for realism and view-consistency (Lin et al., 20 Sep 2025).

In the third stage, Action Unit–Driven Neural Face Rigging uses Neural Face Rigging (NFR) to deform the mesh according to randomly sampled but clinically calibrated AU intensities, with AUs 4, 6, 7, 9, 10, and 43 identified as critical for pain (Lin et al., 20 Sep 2025). The paper states that this provides precise AU control and exact facial deformations for targeted pain expressions and known PSPI labels, surpassing limitations of 2D diffusion and GAN methods which cannot reliably manipulate individual AUs (Lin et al., 20 Sep 2025).

Additional processing steps generate pain-region heatmaps by measuring vertex displacements before and after rigging, and use Kandinsky diffusion models for background inpainting after 3D-to-2D rendering (Lin et al., 20 Sep 2025). The result is not only an RGB corpus but a tightly coupled facial-geometry, AU, PSPI, and spatial-heatmap dataset.

4. ViTPain and cross-modal distillation

ViTPain is the learning framework introduced alongside 3DPain (Lin et al., 20 Sep 2025). It is described as a dual-branch Vision Transformer (ViT-Large)–based model with one branch for multi-class PSPI classification over scores 0–16 and a second branch for regression of the six AUs critical for PSPI (Lin et al., 20 Sep 2025).

The central design is heatmap-supervised cross-modal distillation. A teacher model is trained on generated pain-region heatmaps and serves as a modality specialist for spatial localization and interpretability, while a student model is trained on RGB face images and distilled with the teacher’s latent knowledge (Lin et al., 20 Sep 2025). The source specifies multi-level distillation losses: output KL-divergence for PSPI, mean squared error for AU regression, and mean squared error for CLS token features (Lin et al., 20 Sep 2025).

ViTPain also incorporates AU-specific query tokens and cross-attention. The source characterizes these as learnable query tokens for each AU, enabling highly localized, physiologically interpretable feature extraction for AU intensity regression (Lin et al., 20 Sep 2025). This emphasis on localized AU-specific representations aligns naturally with the dataset’s explicit AU and heatmap supervision.

On the UNBC-McMaster benchmark, ViTPain with 3DPain data and heatmap supervision achieves AUROC $= 0.91$ with 0.54 F1, while versions without 3DPain or heatmap supervision drop to AUROC values from 0.83 to 0.90 (Lin et al., 20 Sep 2025). The same source reports PSPI “$\pm 1$” and “$\pm 2$” tolerance metrics as showing high robustness fitting clinical use, with 0.63–0.83 accuracy within 2 PSPI points (Lin et al., 20 Sep 2025). The paper attributes improvements not only to predictive accuracy but also to interpretability and clinical reliability.

5. Position within automated pain-assessment research

3DPain belongs to a broader trajectory in which automated pain assessment has relied on limited real datasets, facial landmarks, or compact video representations. One geometry-based approach represented facial movement with 66 facial points, Gram matrices, a Riemannian manifold of fixed-rank positive semi-definite matrices, temporal alignment via the Global Alignment Kernel, and Support Vector Regression, obtaining competitive results on UNBC-McMaster (Szczapa et al., 2020). A later extension decomposed the face into jaw, mouth, nose, and eyes, used late fusion across regions, and reported a best 5-fold MAE of 1.36 on UNBC-McMaster and 1.06 on Biovid Heat Pain (Szczapa et al., 2022).

Other work has emphasized compact spatiotemporal encodings or transformer architectures. Adaptive Hierarchical Spatio-temporal Dynamic Imaging encoded facial videos into a single RGB image and reported an MSE of 0.27 on UNBC and 89.76% accuracy on BioVid for pain versus neutral classification (Serraoui et al., 2023). A fully-attentive transformer pipeline trained on 3D-registered and frontalized UNBC faces reported F1 score $0.55 \pm 0.15$ for ViT-1 and $0.55 \pm 0.13$ for ViViT-1, with attention maps focusing on clinically relevant facial regions (Fiorentini et al., 2022).

Within this landscape, 3DPain changes the data regime rather than only the model class. It introduces explicit control over AU activations, facial structure, demographic attributes, and PSPI labels, and augments RGB supervision with pain-region heatmaps (Lin et al., 20 Sep 2025). This distinguishes it from real-data methods whose labels are constrained by the availability, ethics, and imbalance of clinical recordings.

A closely related synthetic-data study for pain videos generated 8,600 synthetic faces by transferring genuine pain expressions onto diverse synthetic avatars. In that study, the mixed synthetic-plus-real training regime achieved AUROC 0.780, F1-score 0.817, and Accuracy 0.708 on real test data, outperforming real-only training, whereas synthetic-only training yielded AUROC 0.581 despite strong F1-score and accuracy (Nasimzada et al., 2024). This is a useful comparison point because it shows that synthetic pain data can measurably improve generalization while still leaving a synth-to-real gap.

6. Interpretive issues, misconceptions, and open directions

A common misconception is that synthetic facial pain data by itself eliminates the need for real clinical validation. The surrounding literature does not support that conclusion. The video-synthesis study cited above reports that the best results were obtained when combining synthetic and real data, and explicitly notes that synth-to-real generalization is not perfect (Nasimzada et al., 2024). This suggests that 3DPain should be interpreted as a controllable and richly annotated foundation for training and pretraining, not as a complete replacement for clinically collected corpora.

A second misconception is that realism alone suffices for clinical relevance. The 3DPain design instead places emphasis on clinically calibrated AU intensities, exact PSPI computation, paired neutral/pain images, and heatmap supervision (Lin et al., 20 Sep 2025). In other words, the dataset’s novelty is not reducible to photorealistic rendering; it lies in the coupling of 3D controllability, annotation richness, and explicit pain coding.

The broader pain-analysis literature also indicates that deployment validity remains population- and task-dependent. Work on unobtrusive pain monitoring in older adults with dementia stressed that this population is not represented in existing facial expression datasets of pain and validated a system directly on a dementia cohort (Rezaei et al., 2021). A plausible implication is that demographic balancing in 3DPain addresses one source of bias, but specialized validation on non-communicative or clinically atypical populations remains necessary.

Taken together, 3DPain establishes a controllable, diverse, and clinically grounded foundation for generalizable automated pain assessment (Lin et al., 20 Sep 2025). Its significance lies in shifting facial pain analysis toward a regime where AU-level mechanisms, PSPI labels, demographic attributes, and pain-region localization are all jointly available at scale, thereby enabling model classes such as ViTPain that pursue accuracy, interpretability, and clinical reliability within the same training framework.