TrueSkin Dataset and Benchmark

• TrueSkin is a dataset and benchmark that explicitly differentiates a person's inherent skin tone from the apparent tone influenced by lighting and camera settings.
• It comprises 7,299 images across six visually defined classes, combining real and synthetic data curated through consensus to ensure balance and diversity.
• The resource supports both recognition and generation tasks, achieving notable improvements using a supervised baseline and LoRA fine-tuning over large multimodal models.

TrueSkin is a dataset, benchmark, and training resource for true skin tone recognition and generation, introduced to distinguish a person’s inherent skin tone from the apparent skin tone induced by lighting, camera settings, shadows, reflections, and image processing. It was designed to expose systematic failure modes in large multimodal models and text-to-image generators, and to provide a more controlled basis for improving fairness and fidelity in skin-tone-related vision tasks. In the paper that introduced it, TrueSkin contains 7,299 images categorized into 6 visually defined classes—dark, brown, tan, medium, light, and pale—and is used both to benchmark existing systems and to train improved recognition and generation models (Lu, 13 Sep 2025).

1. Conceptual basis and label space

The organizing concept of TrueSkin is the distinction between true skin tone and apparent skin tone. True skin tone is defined as a person’s inherent skin tone independent of external factors such as lighting, camera settings, or image processing. Apparent skin tone, by contrast, can vary markedly with illumination, color cast, shadows, reflections, or capture conditions. The dataset and benchmark are built around the claim that many existing systems conflate the two, especially when operating on uncontrolled images in the wild.

TrueSkin adopts a six-category structure inspired by the cardinality of the Fitzpatrick scale, but it does not use Fitzpatrick’s medical criteria such as burning or tanning response to ultraviolet light. Instead, it defines the categories purely by visual perception, using natural-language descriptions and example images. The six classes are as follows.

Class	Visual definition
Dark	Deepest skin tone, from deep brown to nearly black
Brown	Deep skin tone but not extremely dark
Tan	Lighter brown, often golden in sunlight
Medium	Neutral and balanced tone
Light	Creamy to fair, with visible warmth
Pale	Lightest tone, cooler than light skin

The paper explicitly rejects finer-grained taxonomies such as the Monk scale and von Luschan scale for this dataset construction, reporting that when those scales were attempted, less than 30% of samples achieved valid consensus. This motivates the six-class TrueSkin label space as a compromise between granularity and annotation consistency.

2. Dataset construction, composition, and annotation

TrueSkin combines real images and synthetic images. The real images were sourced from Human Faces on Kaggle, and the synthetic images were generated using FLUX.1-dev. The synthetic component is not treated as automatically reliable; rather, it is used to fill category gaps after curation and re-annotation (Lu, 13 Sep 2025).

The dataset totals 7,299 images. Class totals are:

• Dark: 1,701
• Brown: 1,498
• Tan: 1,336
• Medium: 1,004
• Light: 1,152
• Pale: 608

Its real subset contains 138 dark, 247 brown, 413 tan, 387 medium, 485 light, and 120 pale images. Its synthetic subset contains 1,563 dark, 1,251 brown, 923 tan, 617 medium, 667 light, and 488 pale images. This mixed construction is central to the paper’s balancing strategy.

Annotation is consensus-based. Six annotators from diverse ethnic backgrounds labeled the images, and a sample was included only if at least four annotators reached a consensus. The protocol used example images and detailed natural-language descriptions, and the example set was expanded during annotation to reduce discrepancies. The paper emphasizes three design principles: consistency, balance, and diversity.

The dataset was explicitly designed to cover diverse conditions, including varying lighting color, lighting intensity, camera angles, capture settings, close-ups, full-body images, non-facial images, varying proportions of visible skin, and a broad age range from infants to the elderly. It also includes many images in which skin occupies less than 10% of the image, so the recognition task cannot be reduced to simple global color statistics.

The authors quantify balance using Coefficient of Variation (CV) and Kullback–Leibler (KL) divergence, with lower values interpreted as more uniform class distributions.

Dataset	CV	KL Divergence
Fitzpatrick17k	0.5566	0.2013
SCIN	0.7750	0.3926
TrueSkin (real only)	0.5106	0.1680
TrueSkin	0.3179	0.0652

These values are used to argue that the addition of curated synthetic data makes TrueSkin more balanced than both prior datasets and its own real-only subset.

3. Recognition benchmark and supervised baseline

The recognition task is defined as six-way classification over the TrueSkin taxonomy. The input is an in-the-wild human image, and the output is one of $$\{\text{dark}, \text{brown}, \text{tan}, \text{medium}, \text{light}, \text{pale}\}$$. The paper treats the label space as ordinal, mapping the classes to indices

$$\text{dark}=0,\ \text{brown}=1,\ \text{tan}=2,\ \text{medium}=3,\ \text{light}=4,\ \text{pale}=5.$$

This allows evaluation not only by exact accuracy but also by the severity of misclassification through mean squared error (MSE) over class indices (Lu, 13 Sep 2025).

The recognition benchmark includes Llama 3.2, LLaVA-NeXT, Janus-Pro-7B, Qwen2.5-VL, Phi-3.5, and a conventional ITA-based baseline. Because large multimodal models tend to answer verbosely, the paper evaluates three prompting strategies and ultimately uses the prompt:

“You are given an image of a person. Determine the person's skin tone and use only one word as your response without any additional explanation.”

For supervised training, the paper introduces a baseline classifier based on EfficientNet-B1 pretrained on ImageNet, with the final layer replaced by a $1280 \times 6$ linear layer. To reflect the ordinal structure of the labels, it defines a weighted cross-entropy in which the per-sample, per-class weight is

$w_{n,j} = \lambda e^{|y_n-j|}.$

The dataset split for this model is 80% training, 10% validation, and 10% test, with training run for 10 epochs using AdamW at learning rate $1\times10^{-4}$ and batch size 32.

Recognition results on TrueSkin are:

Model	Accuracy	MSE
Llama3.2	44.31%	0.9866
LLaVA-NeXT	40.45%	1.3501
Janus-Pro-7B	48.83%	0.8880
Qwen2.5	43.12%	1.0135
Phi-3.5	41.40%	1.1895
ITA	24.84%	5.8816
TrueSkin baseline	74.18%	0.3374

The strongest large multimodal model in this benchmark is Janus-Pro-7B at 48.83%, whereas the supervised TrueSkin baseline reaches 74.18%, a gain of 25.35 percentage points. The baseline also reports only 2.16% of predictions deviating by more than one class level. On Fitzpatrick17k, the trained TrueSkin model is reported to achieve 30.61% accuracy and 78.85% within one level of deviation, compared with 26.72% accuracy and 60.34% within one level for the neural classifier reported in the Fitzpatrick17k paper.

A central empirical claim of the paper is that large multimodal models often misclassify intermediate tones and exhibit a bias toward lighter predictions, whereas the ITA baseline fails badly because it estimates apparent skin color rather than true skin tone under diverse illumination.

4. Generation benchmark and LoRA fine-tuning

TrueSkin also defines a generation task: given a text prompt containing an explicit target such as “[target tone] skin”, produce an image whose rendered skin tone matches that target. Prompts were generated using ChatGPT-4o and varied hairstyle, lighting, clothing, background, and camera distance. For each skin tone × gender combination, the benchmark uses 50 unique prompts and 2 images per prompt, yielding 200 samples per skin tone per model (Lu, 13 Sep 2025).

Generated outputs are annotated using the same procedure as the dataset itself. The primary metrics are accuracy and MSE over the same ordinal label mapping used in recognition.

The benchmarked generators are Stable Diffusion XL (SDXL), Stable Diffusion 3 Large-Turbo, and FLUX.1-dev. Results are:

Model	Accuracy	MSE
FLUX.1-dev	56.08%	1.1208
SD3 Large Turbo	57.50%	0.9967
SDXL (Step 0)	61.08%	0.6008
SDXL (Step 400)	63.08%	0.5142
SDXL (Step 800)	64.75%	0.4800

The paper fine-tunes SDXL using LoRA with rank 16, two A100 GPUs, batch size 4, and learning rate $1\times10^{-4}$. Prompts for real images are generated by JoyCaption with explicit skin tone descriptions, while prompts for generated images are reused from the original generation process. Under this setup, SDXL improves from 61.08% to 64.75% accuracy, while MSE drops from 0.6008 to 0.4800.

A distinctive result concerns prompt-attribute interference. For FLUX.1-dev, prompts involving snow or nighttime raise the success rate for pale skin generation to 82.5% (33/40), compared with an overall pale-skin success rate of 24.5% (49/200). The paper presents this as direct evidence that unrelated contextual descriptors can override explicit skin-tone instructions.

5. Relation to fairness-oriented skin-tone research

TrueSkin is part of a broader shift in computer vision from proxy labels and purely colorimetric heuristics toward explicitly benchmarked, perceptually grounded skin-tone representations. Its emphasis on separating true skin tone from apparent image color addresses the same measurement problem that motivates TrustSkin, a fairness pipeline for facial affect analysis across skin tone. TrustSkin argues that fairness conclusions depend materially on how skin tone is operationalized, comparing ITA with a perceptually grounded $H^*$-$L^*$ method and showing that the choice of metric can change subgroup fairness conclusions (Cabanas et al., 27 May 2025).

A parallel development is the STW benchmark, which introduces a 42,313-image dataset labeled with the 10-tone Monk Skin Tone (MST) scale and argues for identity-disjoint evaluation as a safeguard against train–test leakage. STW also reports that classical color-based pipelines perform poorly in the wild and proposes SkinToneNet, a fully fine-tuned ViT-Small, for out-of-domain skin-tone auditing (Matias et al., 2 Mar 2026). Relative to STW, TrueSkin differs in two important ways: it uses a six-class visually defined taxonomy rather than MST-10, and it treats generation fidelity as a first-class benchmark alongside recognition.

This suggests a division of labor across recent work. TrustSkin is primarily a fairness-auditing pipeline for downstream facial analysis; STW is a large-scale classification benchmark for skin tone in the wild; TrueSkin is a dataset-benchmark-training resource focused specifically on true skin tone recognition and generation. All three, however, are organized around the view that naïve image color is not an adequate proxy for robust, fair, or controllable skin-tone modeling.

6. Limitations, scope, and prospective extensions

The paper states two principal limitations of TrueSkin. First, the labels are coarse-grained: six classes cannot capture finer distinctions such as yellowish versus reddish tones. Second, annotation retains residual subjectivity even after six-annotator consensus filtering (Lu, 13 Sep 2025).

Additional constraints are structural. The balancing strategy depends heavily on synthetic data; the paper does not provide detailed metadata for camera devices, sensor models, geographic provenance, or explicit demographic breakdown by race, ethnicity, or gender. It also does not present a formal governance section beyond source references. The dataset is motivated partly by healthcare disparities, but it is not a clinical skin-health dataset, and it is not a patient-specific digital twin.

The authors propose several future directions: expanding dataset scale and diversity, developing automated or semi-automated annotation tools, using multimodal or self-supervised models to reduce human subjectivity, and integrating causal modeling to disentangle skin tone from environmental factors. These proposals follow directly from the paper’s core diagnosis: current failures arise not only from weak classifiers or generators, but from inadequate supervision for separating inherent skin tone from nuisance image conditions.

In that sense, TrueSkin is best understood not as a single algorithm but as a research infrastructure for a specific problem formulation. Its broader significance lies in making skin tone a measurable, trainable, and auditable target in both discriminative and generative vision systems.