SCUT-FBP5500: Facial Beauty Prediction Dataset

• SCUT-FBP5500 is a comprehensive benchmark dataset for facial beauty prediction that includes 5,500 frontal face images annotated with beauty scores and 86 facial landmarks.
• It supports multiple computational paradigms—classification, regression, and ranking—by offering diverse demographic subsets and both appearance and geometric labels.
• While enabling state-of-the-art deep learning methods, the dataset also highlights challenges such as ethnic imbalances and the subjectivity inherent in beauty ratings.

Searching arXiv for SCUT-FBP5500 and related papers to ground the article. SCUT-FBP5500 is a benchmark dataset for facial beauty prediction (FBP), introduced to support multiple computational paradigms—classification, regression, and ranking—over a relatively large set of annotated face images with both appearance and geometric labels. It contains 5,500 frontal faces with diverse properties, including male and female subjects, Asian and Caucasian subjects, and varied ages, together with beauty scores within $[1,5]$, beauty score distributions, and dense facial landmarks (Liang et al., 2018). Within the FBP literature it has become a central benchmark for deep learning systems, but later work has also treated it as a cautionary case: despite its structured design and multi-rater labeling, models trained on it can exhibit significant ethnic disparities and limited fairness under cross-dataset evaluation (Hamel et al., 29 Sep 2025).

1. Dataset constitution and original design goals

SCUT-FBP5500 was proposed as a “diverse benchmark dataset for multi-paradigm facial beauty prediction” with the explicit aim of overcoming the narrowness of earlier FBP datasets, which were often tied to a single computation paradigm or a single feature type (Liang et al., 2018). The dataset comprises 5,500 frontal, un-occluded faces with neutral expression, ages roughly 15–60, collected from Internet sources, DataTang, and the 10k US Adult Faces dataset. Its four subsets are Asian Female (AF), Asian Male (AM), Caucasian Female (CF), and Caucasian Male (CM), with counts of 2,000, 2,000, 750, and 750 images respectively (Liang et al., 2018).

This composition makes two structural facts simultaneously true. First, the dataset is balanced by gender at the global level, with 2,750 female and 2,750 male faces. Second, it is not balanced by ethnicity: Asian faces number 4,000, whereas Caucasian faces number 1,500 (Liang et al., 2018). Later fairness analysis explicitly identifies this Asian/Caucasian dichotomy and skew as a potential source of sampling bias (Hamel et al., 29 Sep 2025).

The original design emphasized versatility. SCUT-FBP5500 provides a per-face average beauty score in $[1,5]$, the full distribution of individual ratings, and 86 facial landmarks, thereby enabling appearance-based and shape-based methods, as well as subgroup-specific or unified models (Liang et al., 2018). This multi-paradigm intent distinguishes it from narrower predecessors such as the earlier 500-image SCUT-FBP, which focused on Asian female faces only (Liang et al., 2018).

2. Annotation protocol and label structure

Each image in SCUT-FBP5500 is assigned a beauty score on a $[1,5]$ scale by 60 volunteers; the main scalar label is the average over those 60 ratings (Liang et al., 2018). The volunteers were aged 18–27, with mean age 21.6, and labeling was conducted through a web-based GUI deployed on Ali Cloud. The four demographic subsets were labeled separately, with faces shown randomly to each volunteer (Liang et al., 2018).

The dataset incorporated an explicit consistency check. About 10% of faces were repeated, and for a repeated face, if the correlation coefficient between the two scoring vectors was less than 0.7 for that volunteer, the volunteer was asked to re-rate the face to decide the final score (Liang et al., 2018). For analysis of score distributions, ratings differing from the face average by more than 2 points were marked as outliers. Reported outlier ratios were low: 0.3% for CF, 0.4% for CM, 0.3% for AF, and 0.4% for AM (Liang et al., 2018).

Beyond the scalar average, the full score distribution is retained. This supports label distribution learning, modeling of rating uncertainty, and personalized preference analysis (Liang et al., 2018). The original analysis reports that, for each of AF, AM, CF, and CM, the histogram of average scores is well described by a mixture of two Gaussians, while the standard deviation of scores across raters is roughly Gaussian, with most standard deviations in the range $[0.6, 0.7]$ (Liang et al., 2018).

The geometric annotation layer is also unusually rich. Every image has 86 facial landmarks over eyes, eyebrows, nose, mouth, and face contour. These were initialized by an Active Shape Model trained on the earlier SCUT-FBP dataset and then manually corrected through a custom landmark GUI (Liang et al., 2018). As a result, the dataset is usable not only for end-to-end image regression, but also for geometric feature extraction, PCA-based shape analysis, and hybrid appearance-plus-shape systems.

A recurrent point in later work is that these labels remain subjective even when heavily averaged. Some papers treat the mean over 60 ratings as a statistically robust scalar target (Boukhari, 17 Oct 2025), while later fairness-oriented work argues that unknown annotator demographics can still encode cultural preferences into the labels and thereby into trained models (Hamel et al., 29 Sep 2025). This suggests that the averaging protocol reduces noise without eliminating value-laden bias.

3. Computational paradigms and benchmark protocols

From its inception, SCUT-FBP5500 was intended to support classification, regression, and ranking formulations of FBP (Liang et al., 2018). In regression, the target is the continuous score $y_i$, with common objectives including

$$\text{MSE} = \frac{1}{N}\sum_{i=1}^N (y_i - \hat{y}_i)^2$$

and

$$\text{MAE} = \frac{1}{N}\sum_{i=1}^N |y_i - \hat{y}_i|.$$

For evaluation, the benchmark literature consistently reports Pearson correlation coefficient (PC), MAE, and RMSE (Liang et al., 2018).

The original paper established baseline protocols that are still influential. Shallow geometric and Gabor-feature regressors were typically evaluated with 10-fold cross validation, whereas deep CNN baselines used both 5-fold cross validation and a 60%/40% train/test split (Liang et al., 2018). Under 5-fold cross validation, the original deep baselines reported approximately: AlexNet with PC $0.8634$, MAE $0.2651$, RMSE $0.3481$; ResNet-18 with PC $[1,5]$0, MAE $[1,5]$1, RMSE $[1,5]$2; and ResNeXt-50 with PC $[1,5]$3, MAE $[1,5]$4, RMSE $[1,5]$5 (Liang et al., 2018).

The dataset also proved well suited to hybrid loss formulations. “ComboLoss for Facial Attractiveness Analysis with Squeeze-and-Excitation Networks” formulates attractiveness prediction as simultaneous scalar regression, discretized classification, and expectation-based regression, combining

$[1,5]$6

with $[1,5]$7, $[1,5]$8, and $[1,5]$9 (Xu et al., 2020). On SCUT-FBP5500, that work reports 5-fold-cross-validation performance of MAE $[1,5]$0, RMSE $[1,5]$1, and PC $[1,5]$2, surpassing earlier baselines and showing that loss design alone materially changes benchmark standings (Xu et al., 2020).

A practical complication is that later papers do not all use the same evaluation convention. Some average over all five folds, some report a single designated fold, and some use the official cross_validation_5 split as a fixed train/test partition. Consequently, SCUT-FBP5500 is a stable benchmark in name, but not always in protocol.

4. Deep-learning benchmark evolution

Recent work has used SCUT-FBP5500 as the sole benchmark for increasingly specialized architectures, often treating the task as continuous regression from a $[1,5]$3 face image to a scalar beauty score. The dataset has therefore become a testbed for architectural questions about global facial structure, local texture modeling, state-space modules, and generative priors.

Method	Reported protocol	Reported metrics
ComboLoss + SEResNeXt-50	5-fold cross validation	PC 0.9199, MAE 0.2050, RMSE 0.2704
SIT	standard 5-fold protocol, experiments on the third fold	PC 0.9187, MAE 0.2180, RMSE 0.2760
Mamba-CNN	official cross-validation split / official test split	PC 0.9187, MAE 0.2022, RMSE 0.2610
VM-BeautyNet	standard 5-fold cross validation, averaged over 5 folds	PC 0.9212, MAE 0.2085, RMSE 0.2698
Diff-FBP	standard 5-fold cross validation, averaged over 5 folds	body table: PCC 0.9220, MAE 0.2110
MD-Net	official cross_validation_5 split	PC 0.9235, MAE 0.2006, RMSE 0.2580

The underlying model families differ substantially. SIT uses a MobileNetV2 backbone, three parallel convolution branches at $[1,5]$4, $[1,5]$5, and $[1,5]$6, and a Transformer encoder that models interactions between the resulting scale tokens, arguing that explicit inter-scale interaction improves performance (Boukhari, 5 Sep 2025). Mamba-CNN instead introduces a Mamba-inspired gating mechanism into a hierarchical convolutional backbone and combines it with a multi-scale feature pyramid, seeking better long-range modulation without quadratic self-attention cost (Boukhari, 1 Sep 2025). VM-BeautyNet uses a dual-branch ensemble with a ViT branch and a Vision Mamba branch, fused by a learnable linear module, and reports that ViT captures global aesthetics whereas Mamba captures detailed refinement (Boukhari, 17 Oct 2025).

Another line of work replaces conventional discriminative pretraining with generative or hybrid priors. Diff-FBP pre-trains a Diffusion Transformer on FFHQ through a denoising objective, freezes the encoder, and learns only a small regression head on SCUT-FBP5500; the paper’s abstract highlights PCC $[1,5]$7, whereas its body comparison table reports PCC $[1,5]$8 and MAE $[1,5]$9 under 5-fold cross validation (Boukhari et al., 27 Jul 2025). MD-Net uses a frozen Stable Diffusion v1.5 U-Net encoder for local aesthetic descriptors, a Vision Mamba stream for global structure, and cross-attention fusion, reporting PC $[0.6, 0.7]$0, MAE $[0.6, 0.7]$1, and RMSE $[0.6, 0.7]$2 on the official split (Boukhari, 21 Sep 2025).

These results are often described as state of the art within their respective papers, but direct comparison requires caution because protocol definitions differ. Some results are averages across all five folds, some are single-fold results, and some use a fixed official split. A plausible implication is that SCUT-FBP5500 serves not merely as a leaderboard benchmark but as a methodological microscope: small protocol changes and small architectural changes can both materially alter reported performance.

5. Fairness analysis and bias controversies

A central misconception about SCUT-FBP5500 is that multi-rater averaging and a structured subgroup design make it neutral. Later work directly disputes this. “Analysis of Bias in Deep Learning Facial Beauty Regressors” treats SCUT-FBP5500 as a core training dataset and a central example of how “seemingly balanced” beauty data can still yield biased deep learning models (Hamel et al., 29 Sep 2025).

That study emphasizes several structural features of the dataset. SCUT-FBP5500 contains only two ethnic groups, Asian and Caucasian, and Asian faces are heavily overrepresented. The fairness analysis aggregates Asian females and males, and Caucasian females and males, when assessing ethnic disparity (Hamel et al., 29 Sep 2025). In that work, beauty scores from SCUT-FBP5500 and MEBeauty are normalized to $[0.6, 0.7]$3 before training via

$[0.6, 0.7]$4

Models are then evaluated not only in-domain but also across datasets and on FairFace, which provides balanced race labels but no beauty annotations (Hamel et al., 29 Sep 2025).

The reported disparities are statistically strong. Across ethnic groups, both models exhibit significant prediction disparities with $[0.6, 0.7]$5, even when evaluated on FairFace (Hamel et al., 29 Sep 2025). When the MEBeauty-trained model is evaluated on SCUT-FBP5500, mean predicted beauty scores are close for Asian and Caucasian faces—Asian mean prediction approximately $[0.6, 0.7]$6, Caucasian approximately $[0.6, 0.7]$7—but error distributions differ much more sharply, with Asian mean error approximately $[0.6, 0.7]$8 versus Caucasian approximately $[0.6, 0.7]$9 (Hamel et al., 29 Sep 2025). The associated Mann–Whitney U tests and Kolmogorov–Smirnov tests show statistically significant differences for both prediction and error distributions, especially the errors.

The same paper formalizes fairness in regression via distributional parity and error parity. Distributional parity requires that group-wise prediction distributions be statistically indistinguishable, expressed as

$y_i$0

as judged by non-parametric tests such as Mann–Whitney U or Kruskal–Wallis (Hamel et al., 29 Sep 2025). On FairFace, the fairness outcome is poor: for the SCUT-trained model, only 1 of 21 pairwise racial comparisons, or 4.8%, satisfies distributional parity; for the MEBeauty-trained model, 2 of 21, or 9.5%, do so (Hamel et al., 29 Sep 2025). The paper interprets the cross-dataset behavior as algorithmic amplification of societal beauty biases rather than mitigation.

This has two consequences for how SCUT-FBP5500 is interpreted. First, high aggregate accuracy does not imply equitable behavior. The SCUT-trained model in that study obtains test MSE $y_i$1 on held-out SCUT data, yet still produces systematic disparities across racial groups when transferred to MEBeauty and FairFace (Hamel et al., 29 Sep 2025). Second, the benchmark’s apparent cleanliness—high-quality frontal faces, repeated ratings, averaged labels—does not resolve deeper questions about whose aesthetic preferences the labels encode.

6. Applications, significance, and enduring limitations

The original SCUT-FBP5500 paper situates FBP within applications such as makeup synthesis and recommendation, content-based image retrieval, aesthetic or plastic surgery support, and automatic face beautification and enhancement (Liang et al., 2018). Because the dataset includes both raw images and 86-point geometry, it supports a wide spectrum of research agendas: image-based regression, shape-based inference, ranking, label distribution learning, subgroup-specific models, and multimodal fusion.

Its enduring significance lies in that breadth. SCUT-FBP5500 is large enough to support modern deep baselines, rich enough to support uncertainty-aware and geometry-aware models, and standardized enough to support reproducible benchmarking (Liang et al., 2018). That is why it remains the primary or sole benchmark in diverse later studies, from loss-function engineering (Xu et al., 2020) to hybrid CNN–Transformer designs (Boukhari, 5 Sep 2025), ViT–Mamba ensembling (Boukhari, 17 Oct 2025), state-space gating (Boukhari, 1 Sep 2025), diffusion-based generative priors (Boukhari et al., 27 Jul 2025), and diffusion-plus-Mamba fusion (Boukhari, 21 Sep 2025).

At the same time, its limitations are now part of its scholarly identity. The dataset is modest by contemporary deep learning standards; its faces are constrained to frontal, neutral-expression, un-occluded images; its ethnicity structure is binary and imbalanced; and its beauty labels remain subjective averages rather than universal truths (Liang et al., 2018). Later papers explicitly acknowledge that models trained on it can inherit biases related to race, gender, and cultural beauty standards, and that fairness analysis is not optional for deployment-oriented work (Hamel et al., 29 Sep 2025).

A balanced assessment is therefore dual. SCUT-FBP5500 is both a foundational FBP benchmark and a case study in the limits of benchmark-centered progress. It has enabled the field to formalize facial beauty prediction as a rigorous machine learning problem with shared protocols and metrics, but it also shows that high benchmark performance can coexist with unstable cross-protocol comparability and substantial demographic bias. In current research practice, the dataset is best understood not as a neutral ground truth for attractiveness, but as an influential, richly annotated, and methodologically productive artifact whose assumptions require continual scrutiny.