---
title: 'SCUT-FBP5500: Facial Beauty Prediction Dataset'
url: https://www.emergentmind.com/topics/scut-fbp5500
type: topic
---

# SCUT-FBP5500: Facial Beauty Prediction Dataset

Searching arXiv for SCUT-FBP5500 and related papers to ground the article.
SCUT-FBP5500 is a benchmark dataset for facial beauty prediction (FBP), introduced to support multiple computational paradigms—classification, regression, and ranking—over a relatively large set of annotated face images with both appearance and geometric labels. It contains 5,500 frontal faces with diverse properties, including male and female subjects, Asian and Caucasian subjects, and varied ages, together with beauty scores within \([1,5]\), beauty score distributions, and dense facial landmarks [1801.06345]. Within the FBP literature it has become a central benchmark for deep learning systems, but later work has also treated it as a cautionary case: despite its structured design and multi-rater labeling, models trained on it can exhibit significant ethnic disparities and limited fairness under cross-dataset evaluation [2509.24138].

## 1. Dataset constitution and original design goals

SCUT-FBP5500 was proposed as a “diverse benchmark dataset for multi-paradigm facial beauty prediction” with the explicit aim of overcoming the narrowness of earlier FBP datasets, which were often tied to a single computation paradigm or a single feature type [1801.06345]. The dataset comprises 5,500 frontal, un-occluded faces with neutral expression, ages roughly 15–60, collected from Internet sources, DataTang, and the 10k US Adult Faces dataset. Its four subsets are Asian Female (AF), Asian Male (AM), Caucasian Female (CF), and Caucasian Male (CM), with counts of 2,000, 2,000, 750, and 750 images respectively [1801.06345].

This composition makes two structural facts simultaneously true. First, the dataset is balanced by gender at the global level, with 2,750 female and 2,750 male faces. Second, it is not balanced by ethnicity: Asian faces number 4,000, whereas Caucasian faces number 1,500 [1801.06345]. Later fairness analysis explicitly identifies this Asian/Caucasian dichotomy and skew as a potential source of sampling bias [2509.24138].

The original design emphasized versatility. SCUT-FBP5500 provides a per-face average beauty score in \([1,5]\), the full distribution of individual ratings, and 86 facial landmarks, thereby enabling appearance-based and shape-based methods, as well as subgroup-specific or unified models [1801.06345]. This multi-paradigm intent distinguishes it from narrower predecessors such as the earlier 500-image SCUT-FBP, which focused on Asian female faces only [1801.06345].

## 2. Annotation protocol and label structure

Each image in SCUT-FBP5500 is assigned a beauty score on a \([1,5]\) scale by 60 volunteers; the main scalar label is the average over those 60 ratings [1801.06345]. The volunteers were aged 18–27, with mean age 21.6, and labeling was conducted through a web-based GUI deployed on Ali Cloud. The four demographic subsets were labeled separately, with faces shown randomly to each volunteer [1801.06345].

The dataset incorporated an explicit consistency check. About 10% of faces were repeated, and for a repeated face, if the correlation coefficient between the two scoring vectors was less than 0.7 for that volunteer, the volunteer was asked to re-rate the face to decide the final score [1801.06345]. For analysis of score distributions, ratings differing from the face average by more than 2 points were marked as outliers. Reported outlier ratios were low: 0.3% for CF, 0.4% for CM, 0.3% for AF, and 0.4% for AM [1801.06345].

Beyond the scalar average, the full score distribution is retained. This supports label distribution learning, modeling of rating uncertainty, and personalized preference analysis [1801.06345]. The original analysis reports that, for each of AF, AM, CF, and CM, the histogram of average scores is well described by a mixture of two Gaussians, while the standard deviation of scores across raters is roughly Gaussian, with most standard deviations in the range \([0.6, 0.7]\) [1801.06345].

The geometric annotation layer is also unusually rich. Every image has 86 facial landmarks over eyes, eyebrows, nose, mouth, and face contour. These were initialized by an Active Shape Model trained on the earlier SCUT-FBP dataset and then manually corrected through a custom landmark GUI [1801.06345]. As a result, the dataset is usable not only for end-to-end image regression, but also for geometric feature extraction, PCA-based shape analysis, and hybrid appearance-plus-shape systems.

A recurrent point in later work is that these labels remain subjective even when heavily averaged. Some papers treat the mean over 60 ratings as a statistically robust scalar target [2510.16220], while later fairness-oriented work argues that unknown annotator demographics can still encode cultural preferences into the labels and thereby into trained models [2509.24138]. This suggests that the averaging protocol reduces noise without eliminating value-laden bias.

## 3. Computational paradigms and benchmark protocols

From its inception, SCUT-FBP5500 was intended to support classification, regression, and ranking formulations of FBP [1801.06345]. In regression, the target is the continuous score \(y_i\), with common objectives including
\[
\text{MSE} = \frac{1}{N}\sum_{i=1}^N (y_i - \hat{y}_i)^2
\]
and
\[
\text{MAE} = \frac{1}{N}\sum_{i=1}^N |y_i - \hat{y}_i|.
\]
For evaluation, the benchmark literature consistently reports Pearson correlation coefficient (PC), MAE, and RMSE [1801.06345].

The original paper established baseline protocols that are still influential. Shallow geometric and Gabor-feature regressors were typically evaluated with 10-fold cross validation, whereas deep CNN baselines used both 5-fold cross validation and a 60%/40% train/test split [1801.06345]. Under 5-fold cross validation, the original deep baselines reported approximately: AlexNet with PC \(0.8634\), MAE \(0.2651\), RMSE \(0.3481\); ResNet-18 with PC \(0.8900\), MAE \(0.2419\), RMSE \(0.3166\); and ResNeXt-50 with PC \(0.8997\), MAE \(0.2291\), RMSE \(0.3017\) [1801.06345].

The dataset also proved well suited to hybrid loss formulations. “ComboLoss for Facial Attractiveness Analysis with Squeeze-and-Excitation Networks” formulates attractiveness prediction as simultaneous scalar regression, discretized classification, and expectation-based regression, combining
\[
\mathcal{L}_{\mathrm{combo}} = \alpha L_{\mathrm{reg}} + \beta L_{\mathrm{exp}} + \gamma L_{\mathrm{cls}}
\]
with \(\alpha = 2\), \(\beta = 1\), and \(\gamma = 1\) [2010.10721]. On SCUT-FBP5500, that work reports 5-fold-cross-validation performance of MAE \(0.2050\), RMSE \(0.2704\), and PC \(0.9199\), surpassing earlier baselines and showing that loss design alone materially changes benchmark standings [2010.10721].

A practical complication is that later papers do not all use the same evaluation convention. Some average over all five folds, some report a single designated fold, and some use the official `cross_validation_5` split as a fixed train/test partition. Consequently, SCUT-FBP5500 is a stable benchmark in name, but not always in protocol.

## 4. Deep-learning benchmark evolution

Recent work has used SCUT-FBP5500 as the sole benchmark for increasingly specialized architectures, often treating the task as continuous regression from a \(224 \times 224\) face image to a scalar beauty score. The dataset has therefore become a testbed for architectural questions about global facial structure, local texture modeling, state-space modules, and generative priors.

| Method | Reported protocol | Reported metrics |
|---|---|---|
| ComboLoss + SEResNeXt-50 | 5-fold cross validation | PC 0.9199, MAE 0.2050, RMSE 0.2704 |
| SIT | standard 5-fold protocol, experiments on the third fold | PC 0.9187, MAE 0.2180, RMSE 0.2760 |
| Mamba-CNN | official cross-validation split / official test split | PC 0.9187, MAE 0.2022, RMSE 0.2610 |
| VM-BeautyNet | standard 5-fold cross validation, averaged over 5 folds | PC 0.9212, MAE 0.2085, RMSE 0.2698 |
| Diff-FBP | standard 5-fold cross validation, averaged over 5 folds | body table: PCC 0.9220, MAE 0.2110 |
| MD-Net | official `cross_validation_5` split | PC 0.9235, MAE 0.2006, RMSE 0.2580 |

The underlying model families differ substantially. SIT uses a MobileNetV2 backbone, three parallel convolution branches at \(1\times1\), \(3\times3\), and \(5\times5\), and a Transformer encoder that models interactions between the resulting scale tokens, arguing that explicit inter-scale interaction improves performance [2509.05078]. Mamba-CNN instead introduces a Mamba-inspired gating mechanism into a hierarchical convolutional backbone and combines it with a multi-scale feature pyramid, seeking better long-range modulation without quadratic self-attention cost [2509.01431]. VM-BeautyNet uses a dual-branch ensemble with a ViT branch and a Vision Mamba branch, fused by a learnable linear module, and reports that ViT captures global aesthetics whereas Mamba captures detailed refinement [2510.16220].

Another line of work replaces conventional discriminative pretraining with generative or hybrid priors. Diff-FBP pre-trains a Diffusion Transformer on FFHQ through a denoising objective, freezes the encoder, and learns only a small regression head on SCUT-FBP5500; the paper’s abstract highlights PCC \(0.932\), whereas its body comparison table reports PCC \(0.9220\) and MAE \(0.2110\) under 5-fold cross validation [2507.20363]. MD-Net uses a frozen Stable Diffusion v1.5 U-Net encoder for local aesthetic descriptors, a Vision Mamba stream for global structure, and cross-attention fusion, reporting PC \(0.9235\), MAE \(0.2006\), and RMSE \(0.2580\) on the official split [2509.17172].

These results are often described as state of the art within their respective papers, but direct comparison requires caution because protocol definitions differ. Some results are averages across all five folds, some are single-fold results, and some use a fixed official split. A plausible implication is that SCUT-FBP5500 serves not merely as a leaderboard benchmark but as a methodological microscope: small protocol changes and small architectural changes can both materially alter reported performance.

## 5. Fairness analysis and bias controversies

A central misconception about SCUT-FBP5500 is that multi-rater averaging and a structured subgroup design make it neutral. Later work directly disputes this. “Analysis of Bias in Deep Learning Facial Beauty Regressors” treats SCUT-FBP5500 as a core training dataset and a central example of how “seemingly balanced” beauty data can still yield biased deep learning models [2509.24138].

That study emphasizes several structural features of the dataset. SCUT-FBP5500 contains only two ethnic groups, Asian and Caucasian, and Asian faces are heavily overrepresented. The fairness analysis aggregates Asian females and males, and Caucasian females and males, when assessing ethnic disparity [2509.24138]. In that work, beauty scores from SCUT-FBP5500 and MEBeauty are normalized to \([0,1]\) before training via
\[
y_{\text{norm}} = \frac{y - y_{\min}}{y_{\max} - y_{\min}}.
\]
Models are then evaluated not only in-domain but also across datasets and on FairFace, which provides balanced race labels but no beauty annotations [2509.24138].

The reported disparities are statistically strong. Across ethnic groups, both models exhibit significant prediction disparities with \(p < 0.001\), even when evaluated on FairFace [2509.24138]. When the MEBeauty-trained model is evaluated on SCUT-FBP5500, mean predicted beauty scores are close for Asian and Caucasian faces—Asian mean prediction approximately \(0.5929\), Caucasian approximately \(0.5883\)—but error distributions differ much more sharply, with Asian mean error approximately \(0.0722\) versus Caucasian approximately \(0.0380\) [2509.24138]. The associated Mann–Whitney U tests and Kolmogorov–Smirnov tests show statistically significant differences for both prediction and error distributions, especially the errors.

The same paper formalizes fairness in regression via distributional parity and error parity. Distributional parity requires that group-wise prediction distributions be statistically indistinguishable, expressed as
\[
\hat{Y}_{g_a} \overset{d}{=} \hat{Y}_{g_b},
\]
as judged by non-parametric tests such as Mann–Whitney U or Kruskal–Wallis [2509.24138]. On FairFace, the fairness outcome is poor: for the SCUT-trained model, only 1 of 21 pairwise racial comparisons, or 4.8%, satisfies distributional parity; for the MEBeauty-trained model, 2 of 21, or 9.5%, do so [2509.24138]. The paper interprets the cross-dataset behavior as algorithmic amplification of societal beauty biases rather than mitigation.

This has two consequences for how SCUT-FBP5500 is interpreted. First, high aggregate accuracy does not imply equitable behavior. The SCUT-trained model in that study obtains test MSE \(0.008\) on held-out SCUT data, yet still produces systematic disparities across racial groups when transferred to MEBeauty and FairFace [2509.24138]. Second, the benchmark’s apparent cleanliness—high-quality frontal faces, repeated ratings, averaged labels—does not resolve deeper questions about whose aesthetic preferences the labels encode.

## 6. Applications, significance, and enduring limitations

The original SCUT-FBP5500 paper situates FBP within applications such as makeup synthesis and recommendation, content-based image retrieval, aesthetic or plastic surgery support, and automatic face beautification and enhancement [1801.06345]. Because the dataset includes both raw images and 86-point geometry, it supports a wide spectrum of research agendas: image-based regression, shape-based inference, ranking, label distribution learning, subgroup-specific models, and multimodal fusion.

Its enduring significance lies in that breadth. SCUT-FBP5500 is large enough to support modern deep baselines, rich enough to support uncertainty-aware and geometry-aware models, and standardized enough to support reproducible benchmarking [1801.06345]. That is why it remains the primary or sole benchmark in diverse later studies, from loss-function engineering [2010.10721] to hybrid CNN–Transformer designs [2509.05078], ViT–Mamba ensembling [2510.16220], state-space gating [2509.01431], diffusion-based generative priors [2507.20363], and diffusion-plus-Mamba fusion [2509.17172].

At the same time, its limitations are now part of its scholarly identity. The dataset is modest by contemporary deep learning standards; its faces are constrained to frontal, neutral-expression, un-occluded images; its ethnicity structure is binary and imbalanced; and its beauty labels remain subjective averages rather than universal truths [1801.06345]. Later papers explicitly acknowledge that models trained on it can inherit biases related to race, gender, and cultural beauty standards, and that fairness analysis is not optional for deployment-oriented work [2509.24138].

A balanced assessment is therefore dual. SCUT-FBP5500 is both a foundational FBP benchmark and a case study in the limits of benchmark-centered progress. It has enabled the field to formalize facial beauty prediction as a rigorous machine learning problem with shared protocols and metrics, but it also shows that high benchmark performance can coexist with unstable cross-protocol comparability and substantial demographic bias. In current research practice, the dataset is best understood not as a neutral ground truth for attractiveness, but as an influential, richly annotated, and methodologically productive artifact whose assumptions require continual scrutiny.

Source: https://www.emergentmind.com/topics/scut-fbp5500