---
title: TrueSkin Dataset and Benchmark
url: https://www.emergentmind.com/topics/trueskin
type: topic
---

# TrueSkin Dataset and Benchmark

TrueSkin is a dataset, benchmark, and training resource for **true skin tone recognition and generation**, introduced to distinguish a person’s inherent skin tone from the **apparent skin tone** induced by lighting, camera settings, shadows, reflections, and image processing. It was designed to expose systematic failure modes in large multimodal models and text-to-image generators, and to provide a more controlled basis for improving fairness and fidelity in skin-tone-related vision tasks. In the paper that introduced it, TrueSkin contains **7,299 images** categorized into **6 visually defined classes**—**dark, brown, tan, medium, light, and pale**—and is used both to benchmark existing systems and to train improved recognition and generation models [2509.10980].

## 1. Conceptual basis and label space

The organizing concept of TrueSkin is the distinction between **true skin tone** and **apparent skin tone**. True skin tone is defined as a person’s inherent skin tone independent of external factors such as lighting, camera settings, or image processing. Apparent skin tone, by contrast, can vary markedly with illumination, color cast, shadows, reflections, or capture conditions. The dataset and benchmark are built around the claim that many existing systems conflate the two, especially when operating on uncontrolled images in the wild.

TrueSkin adopts a **six-category structure** inspired by the cardinality of the Fitzpatrick scale, but it does **not** use Fitzpatrick’s medical criteria such as burning or tanning response to ultraviolet light. Instead, it defines the categories purely by visual perception, using natural-language descriptions and example images. The six classes are as follows.

| Class | Visual definition |
|---|---|
| Dark | Deepest skin tone, from deep brown to nearly black |
| Brown | Deep skin tone but not extremely dark |
| Tan | Lighter brown, often golden in sunlight |
| Medium | Neutral and balanced tone |
| Light | Creamy to fair, with visible warmth |
| Pale | Lightest tone, cooler than light skin |

The paper explicitly rejects finer-grained taxonomies such as the **Monk scale** and **von Luschan scale** for this dataset construction, reporting that when those scales were attempted, **less than 30% of samples achieved valid consensus**. This motivates the six-class TrueSkin label space as a compromise between granularity and annotation consistency.

## 2. Dataset construction, composition, and annotation

TrueSkin combines **real images** and **synthetic images**. The real images were sourced from **Human Faces** on Kaggle, and the synthetic images were generated using **FLUX.1-dev**. The synthetic component is not treated as automatically reliable; rather, it is used to fill category gaps after curation and re-annotation [2509.10980].

The dataset totals **7,299 images**. Class totals are:

- **Dark**: 1,701
- **Brown**: 1,498
- **Tan**: 1,336
- **Medium**: 1,004
- **Light**: 1,152
- **Pale**: 608

Its real subset contains **138 dark**, **247 brown**, **413 tan**, **387 medium**, **485 light**, and **120 pale** images. Its synthetic subset contains **1,563 dark**, **1,251 brown**, **923 tan**, **617 medium**, **667 light**, and **488 pale** images. This mixed construction is central to the paper’s balancing strategy.

Annotation is consensus-based. **Six annotators from diverse ethnic backgrounds** labeled the images, and a sample was included only if **at least four annotators reached a consensus**. The protocol used example images and detailed natural-language descriptions, and the example set was expanded during annotation to reduce discrepancies. The paper emphasizes three design principles: **consistency**, **balance**, and **diversity**.

The dataset was explicitly designed to cover diverse conditions, including varying **lighting color**, **lighting intensity**, **camera angles**, **capture settings**, **close-ups**, **full-body images**, **non-facial images**, varying proportions of visible skin, and a broad age range from **infants to the elderly**. It also includes many images in which skin occupies less than 10% of the image, so the recognition task cannot be reduced to simple global color statistics.

The authors quantify balance using **Coefficient of Variation (CV)** and **Kullback–Leibler (KL) divergence**, with lower values interpreted as more uniform class distributions.

| Dataset | CV | KL Divergence |
|---|---:|---:|
| Fitzpatrick17k | 0.5566 | 0.2013 |
| SCIN | 0.7750 | 0.3926 |
| TrueSkin (real only) | 0.5106 | 0.1680 |
| TrueSkin | 0.3179 | 0.0652 |

These values are used to argue that the addition of curated synthetic data makes TrueSkin more balanced than both prior datasets and its own real-only subset.

## 3. Recognition benchmark and supervised baseline

The recognition task is defined as six-way classification over the TrueSkin taxonomy. The input is an in-the-wild human image, and the output is one of \(\{\text{dark}, \text{brown}, \text{tan}, \text{medium}, \text{light}, \text{pale}\}\). The paper treats the label space as **ordinal**, mapping the classes to indices
\[
\text{dark}=0,\ \text{brown}=1,\ \text{tan}=2,\ \text{medium}=3,\ \text{light}=4,\ \text{pale}=5.
\]
This allows evaluation not only by exact accuracy but also by the severity of misclassification through **mean squared error (MSE)** over class indices [2509.10980].

The recognition benchmark includes **Llama 3.2**, **LLaVA-NeXT**, **Janus-Pro-7B**, **Qwen2.5-VL**, **Phi-3.5**, and a conventional **ITA-based** baseline. Because large multimodal models tend to answer verbosely, the paper evaluates three prompting strategies and ultimately uses the prompt:

> “You are given an image of a person. Determine the person's skin tone and use only one word as your response without any additional explanation.”

For supervised training, the paper introduces a baseline classifier based on **EfficientNet-B1** pretrained on **ImageNet**, with the final layer replaced by a **\(1280 \times 6\)** linear layer. To reflect the ordinal structure of the labels, it defines a weighted cross-entropy in which the per-sample, per-class weight is
\[
w_{n,j} = \lambda e^{|y_n-j|}.
\]
The dataset split for this model is **80% training**, **10% validation**, and **10% test**, with training run for **10 epochs** using **AdamW** at learning rate **\(1\times10^{-4}\)** and batch size **32**.

Recognition results on TrueSkin are:

| Model | Accuracy | MSE |
|---|---:|---:|
| Llama3.2 | 44.31% | 0.9866 |
| LLaVA-NeXT | 40.45% | 1.3501 |
| Janus-Pro-7B | 48.83% | 0.8880 |
| Qwen2.5 | 43.12% | 1.0135 |
| Phi-3.5 | 41.40% | 1.1895 |
| ITA | 24.84% | 5.8816 |
| TrueSkin baseline | 74.18% | 0.3374 |

The strongest large multimodal model in this benchmark is **Janus-Pro-7B** at **48.83%**, whereas the supervised TrueSkin baseline reaches **74.18%**, a gain of **25.35 percentage points**. The baseline also reports only **2.16%** of predictions deviating by more than one class level. On **Fitzpatrick17k**, the trained TrueSkin model is reported to achieve **30.61% accuracy** and **78.85% within one level of deviation**, compared with **26.72% accuracy** and **60.34% within one level** for the neural classifier reported in the Fitzpatrick17k paper.

A central empirical claim of the paper is that large multimodal models often misclassify **intermediate tones** and exhibit a bias toward **lighter predictions**, whereas the ITA baseline fails badly because it estimates apparent skin color rather than true skin tone under diverse illumination.

## 4. Generation benchmark and LoRA fine-tuning

TrueSkin also defines a generation task: given a text prompt containing an explicit target such as **“[target tone] skin”**, produce an image whose rendered skin tone matches that target. Prompts were generated using **ChatGPT-4o** and varied **hairstyle**, **lighting**, **clothing**, **background**, and **camera distance**. For each **skin tone × gender** combination, the benchmark uses **50 unique prompts** and **2 images per prompt**, yielding **200 samples per skin tone per model** [2509.10980].

Generated outputs are annotated using the same procedure as the dataset itself. The primary metrics are **accuracy** and **MSE** over the same ordinal label mapping used in recognition.

The benchmarked generators are **Stable Diffusion XL (SDXL)**, **Stable Diffusion 3 Large-Turbo**, and **FLUX.1-dev**. Results are:

| Model | Accuracy | MSE |
|---|---:|---:|
| FLUX.1-dev | 56.08% | 1.1208 |
| SD3 Large Turbo | 57.50% | 0.9967 |
| SDXL (Step 0) | 61.08% | 0.6008 |
| SDXL (Step 400) | 63.08% | 0.5142 |
| SDXL (Step 800) | 64.75% | 0.4800 |

The paper fine-tunes **SDXL** using **LoRA** with rank **16**, **two A100 GPUs**, batch size **4**, and learning rate **\(1\times10^{-4}\)**. Prompts for real images are generated by **JoyCaption** with explicit skin tone descriptions, while prompts for generated images are reused from the original generation process. Under this setup, SDXL improves from **61.08%** to **64.75%** accuracy, while MSE drops from **0.6008** to **0.4800**.

A distinctive result concerns **prompt-attribute interference**. For **FLUX.1-dev**, prompts involving **snow** or **nighttime** raise the success rate for **pale skin** generation to **82.5% (33/40)**, compared with an overall pale-skin success rate of **24.5% (49/200)**. The paper presents this as direct evidence that unrelated contextual descriptors can override explicit skin-tone instructions.

## 5. Relation to fairness-oriented skin-tone research

TrueSkin is part of a broader shift in computer vision from proxy labels and purely colorimetric heuristics toward explicitly benchmarked, perceptually grounded skin-tone representations. Its emphasis on separating true skin tone from apparent image color addresses the same measurement problem that motivates **TrustSkin**, a fairness pipeline for facial affect analysis across skin tone. TrustSkin argues that fairness conclusions depend materially on how skin tone is operationalized, comparing **ITA** with a perceptually grounded **\(H^*\)-\(L^*\)** method and showing that the choice of metric can change subgroup fairness conclusions [2505.20637].

A parallel development is the **STW** benchmark, which introduces a **42,313-image** dataset labeled with the **10-tone Monk Skin Tone (MST)** scale and argues for **identity-disjoint** evaluation as a safeguard against train–test leakage. STW also reports that classical color-based pipelines perform poorly in the wild and proposes **SkinToneNet**, a fully fine-tuned **ViT-Small**, for out-of-domain skin-tone auditing [2603.02475]. Relative to STW, TrueSkin differs in two important ways: it uses a **six-class visually defined taxonomy** rather than MST-10, and it treats **generation fidelity** as a first-class benchmark alongside recognition.

This suggests a division of labor across recent work. TrustSkin is primarily a **fairness-auditing pipeline** for downstream facial analysis; STW is a **large-scale classification benchmark** for skin tone in the wild; TrueSkin is a **dataset-benchmark-training resource** focused specifically on **true skin tone recognition and generation**. All three, however, are organized around the view that naïve image color is not an adequate proxy for robust, fair, or controllable skin-tone modeling.

## 6. Limitations, scope, and prospective extensions

The paper states two principal limitations of TrueSkin. First, the labels are **coarse-grained**: six classes cannot capture finer distinctions such as **yellowish** versus **reddish** tones. Second, annotation retains **residual subjectivity** even after six-annotator consensus filtering [2509.10980].

Additional constraints are structural. The balancing strategy depends heavily on synthetic data; the paper does not provide detailed metadata for **camera devices**, **sensor models**, **geographic provenance**, or explicit **demographic breakdown by race, ethnicity, or gender**. It also does not present a formal governance section beyond source references. The dataset is motivated partly by healthcare disparities, but it is **not a clinical skin-health dataset**, and it is not a patient-specific digital twin.

The authors propose several future directions: expanding dataset **scale** and **diversity**, developing **automated or semi-automated annotation tools**, using **multimodal or self-supervised models** to reduce human subjectivity, and integrating **causal modeling** to disentangle skin tone from environmental factors. These proposals follow directly from the paper’s core diagnosis: current failures arise not only from weak classifiers or generators, but from inadequate supervision for separating inherent skin tone from nuisance image conditions.

In that sense, TrueSkin is best understood not as a single algorithm but as a research infrastructure for a specific problem formulation. Its broader significance lies in making skin tone a **measurable**, **trainable**, and **auditable** target in both discriminative and generative vision systems.

Source: https://www.emergentmind.com/topics/trueskin