---
title: 'StyleBench-H: Perception Benchmark'
url: https://www.emergentmind.com/topics/stylebench-h
type: topic
---

# StyleBench-H: Perception Benchmark

Searching arXiv for the primary paper and closely related work on StyleBench-H and perception-grounded stylization benchmarks.
arXiv search query: "StyleBench-H StyleID stylization-agnostic facial identity recognition"
StyleBench-H is a human-judged benchmark for facial identity verification under stylization, introduced as one component of the StyleID framework for stylization-agnostic facial identity recognition [2604.21689]. It is designed to measure whether an identity encoder’s similarity judgments agree with human same–different decisions when a source portrait is transformed by artistic stylization. In the StyleID formulation, StyleBench-H serves the evaluation role, while StyleBench-S is a separate supervision set derived from psychometric recognition-strength curves and used for calibration and training [2604.21689]. The benchmark targets a specific failure mode of conventional face-recognition systems: models calibrated on natural photographs often treat texture and palette changes as identity drift, or fail to detect identity degradation caused by geometric exaggeration in stylized renderings.

## 1. Definition and scope

StyleBench-H is a dataset of source portrait / stylized portrait pairs used for human perception-grounded identity verification under stylization [2604.21689]. Each datum is a pair \((I_{\mathrm{src}}, I_{\mathrm{style}})\), where \(I_{\mathrm{src}}\) is a real face photograph and \(I_{\mathrm{style}}\) is a stylized rendering. Human annotators answer the question of whether the two images depict the same person. The benchmark thereby operationalizes identity consistency as a perceptual verification problem rather than as a purely feature-space similarity problem.

The benchmark was created because standard identity encoders are typically trained and thresholded on natural face photographs, and these thresholds do not transfer reliably to stylized images [2604.21689]. This suggests that robustness to stylization is not merely a matter of embedding invariance; it also depends on whether the encoder’s score ordering aligns with human perception across varying styles and stylization strengths. A plausible implication is that StyleBench-H occupies a different evaluative regime from ordinary face-verification benchmarks: the objective is not only recognition accuracy, but recognition accuracy under systematic appearance-domain shift.

## 2. Benchmark composition and stylization factors

StyleBench-H covers stylized portraits generated by three controllable stylization frameworks: **IP-Adapter-faceID**, **InstantID**, and **InfiniteYou** [2604.21689]. These include both diffusion-based and flow-matching-based pipelines. The benchmark spans **10 artistic styles** and **7 discrete stylization strengths**, with strength indexed as
\[
s \in \{1/7, 2/7, \ldots, 7/7\}.
\]

The paper also introduces a normalized strength convention in which \(s=0\) corresponds to nearly original identity and \(s=1\) to maximum stylization or potential identity loss [2604.21689]. However, it explicitly notes that the same normalized strength across methods is not perceptually equivalent. Thus, \(s=0.5\) under one stylization method need not produce the same level of identity retention as \(s=0.5\) under another. This is a central design point: StyleBench-H is structured to expose method-dependent perceptual degradation rather than to assume a uniform stylization scale.

The source portraits are sampled from **FFHQ** and filtered to remove large head rotations and images containing more than one person [2604.21689]. This filtering constrains nuisance variation so that identity-preservation failures are more directly attributable to stylization rather than pose or multi-person ambiguity.

## 3. Human annotation protocol

StyleBench-H is explicitly human-judged. The benchmark uses pairwise verification judgments in which participants inspect a source image and a stylized counterpart and decide whether they depict the same person [2604.21689]. The human-study protocol includes several quality-control steps intended to suppress careless or inconsistent annotations.

The participant pool and filtering details reported for StyleBench-H are as follows.

| Aspect | Value |
|---|---|
| Recruited participants | 70 |
| Valid participants after filtering | 68 |
| Mean age | 29.0 |
| Queries per participant | 91 |
| Valid responses collected | 6088 |
| Final valid datapoints | \(N_H = 3551\) |

The participant demographics are reported as **32 male**, **33 female**, and **5 not disclosed** [2604.21689]. Responses faster than image loading time, responses taking longer than 100 seconds, and responses from inconsistent participants identified using a repeated final question were removed. Two participants were discarded, leaving 68 valid participants [2604.21689]. The final benchmark retains only true-positive and true-negative pairs and is balanced between positives and negatives.

This design indicates that StyleBench-H is not merely a collection of stylized face pairs; it is a psychometrically filtered human-perception dataset. A plausible implication is that the benchmark is better interpreted as a calibrated measurement of perceptual identity retention than as a generic verification corpus.

## 4. Generalization splits and evaluation regime

StyleBench-H is used under three generalization settings in StyleID [2604.21689].

### Cross-ID

The test identities are unseen. This split evaluates generalization to new people while keeping the stylization setting within the broader training distribution.

### Cross-Style

This split uses IP-Adapter and evaluates on styles unseen relative to training [2604.21689]. The objective is to test whether a model calibrated on some artistic styles can retain human-aligned identity judgments on new styles.

### Cross-Method

This is the strongest distribution-shift setting. Test identities, stylization methods, and prompts/styles are all unseen, and the split includes two unseen stylization methods: **MTG** and **Flux.2**, each paired with **8 unseen styles** [2604.21689].

These splits make StyleBench-H a benchmark not only for stylization robustness but also for out-of-distribution generalization across identity, style family, and rendering pipeline. This suggests that the benchmark is intended to probe whether an encoder has learned style-agnostic identity structure rather than method-specific calibration heuristics.

## 5. Metrics and benchmark objective

StyleBench-H evaluates agreement between model similarity scores and human verification judgments using standard verification metrics, with emphasis on operating-point behavior [2604.21689]. The key metric highlighted in the main table is **TPR at FPR \(=10^{-2}\)**. The paper also reports **verification accuracy** and **AUROC**, and the appendix includes **TPR@FPR \(=10^{-3}\)** and **TPR@FPR \(=10^{-4}\)**.

The core evaluative question is whether a model’s similarity function remains calibrated under stylization. The paper argues that a single global threshold does not work well across stylization families, because some methods or styles induce much larger perceptual identity drift than others [2604.21689]. This means that photo-domain thresholds can become either too strict or too permissive in stylized domains.

Unlike fully automatic stylization metrics, StyleBench-H is centered on human binary judgments. This distinguishes it from style-similarity or reconstruction-oriented measures commonly used elsewhere in stylization research. Related work on human-aligned aesthetic evaluation similarly emphasizes pairwise human preference and rank-based agreement rather than purely feature-based scoring, although in a different task domain [2501.09012]. StyleBench-H applies that broader philosophy to facial identity under style transformation.

## 6. Relation to StyleBench-S and the StyleID calibration pipeline

StyleBench-H is the evaluation benchmark in a two-part framework. StyleBench-S is the supervision set derived from psychometric recognition-strength curves obtained through controlled **2AFC** experiments [2604.21689]. The workflow described in StyleID is: build StyleBench-H, fit recognition-vs-strength curves from human responses, derive StyleBench-S from those curves, train or calibrate an identity encoder on StyleBench-S, and evaluate on StyleBench-H.

The psychometric component estimates human recognition as a function of stylization strength, method, and style:
\[
P(\text{human recognizes identity} \mid s, m, t).
\]
This formulation captures the fact that identity degradation is not uniform across artistic styles or rendering methods [2604.21689]. StyleBench-H supplies the perceptual endpoint against which encoders are ultimately judged; StyleBench-S converts that perception into a larger training signal.

Within the StyleID model, the calibrated encoder uses a **CLIP-L** backbone with **LoRA** adapters and combines an ArcFace-style angular-margin loss, supervised contrastive loss, and regularization toward frozen CLIP features [2604.21689]. The paper defines the total loss as
\[
\mathcal{L} = \mathcal{L}_{\text{ang}} + \lambda_{\text{scon}} \mathcal{L}_{\text{scon}} + \lambda_{\text{reg}} \mathcal{L}_{\text{reg}},
\]
with \(\lambda_{\text{scon}}=0.6\) and \(\lambda_{\text{reg}}=0.1\) [2604.21689]. Although these equations belong to StyleID rather than to StyleBench-H itself, they clarify the benchmark’s function: StyleBench-H is the perceptual target that motivates calibration.

## 7. Empirical findings and significance

StyleBench-H reveals several systematic patterns in human and model behavior under stylization [2604.21689]. First, human identity recognition decreases as stylization strength increases. Second, the rate of this degradation differs across stylization methods and artistic styles. Third, photo-domain identity encoders such as **ArcFace** and **AdaFace** transfer better than generic encoders such as **CLIP** or **SigLIP2**, but still fail to fully align with human judgments. Fourth, perception-calibrated training improves alignment substantially.

On StyleBench-H, the calibrated StyleID model reports the following **TPR** values [2604.21689].

| Split | StyleID TPR |
|---|---:|
| Cross-ID | 0.9020 |
| Cross-Style | 0.9255 |
| Cross-Method | 0.7444 |

The paper also reports that StyleID achieves the best or near-best accuracy and AUROC on these splits [2604.21689]. In addition, the model generalizes to artist-drawn sketches on **SKSF-A**, with **TPR 0.8891** and **AUROC 0.9922** [2604.21689]. These results suggest that human-calibrated stylization supervision transfers beyond the exact stylization pipelines used in StyleBench-H.

The benchmark also informs downstream applications. When the calibrated StyleID encoder is used as the identity loss in **JoJoGAN**, the paper reports better identity preservation, better style fidelity, fewer artifacts, and stronger human and GPT preference [2604.21689]. This suggests that StyleBench-H is not only diagnostic; it also provides a practical target for improving stylization systems that must preserve identity.

## 8. Conceptual position and limitations

StyleBench-H is best understood as a perception-grounded verification benchmark rather than a generic stylization dataset. Its central claim is that facial identity under stylization cannot be evaluated reliably with photo-domain thresholds or with uncalibrated similarity spaces [2604.21689]. In that sense, it shifts evaluation from “does the embedding remain stable?” to “does the embedding behave like human identity perception?”

Several limitations are either explicit or strongly implied by the benchmark design. The benchmark is specific to facial identity under stylization and does not generalize directly to non-facial content or to broader style-transfer tasks [2604.21689]. Its stylization coverage, while diverse, is tied to the included methods, styles, and controlled strength schedule. Human judgments are binary same–different decisions, which are appropriate for verification but do not capture richer graded notions of resemblance or recognizability. The benchmark also depends on the quality and representativeness of the participant pool and the selected portrait domain.

Even with these constraints, StyleBench-H fills a distinct methodological gap. Other style-oriented benchmarks focus on webpage layout-style consistency [2510.04097], speaking-style control in dialogue [2603.07599], style embeddings for text [2606.31741; 2502.15168], or human-aligned aesthetic reasoning for stylization [2501.09012]. StyleBench-H differs in that its object of evaluation is not style generation quality per se, but identity preservation judged through the lens of human perception under stylization. This suggests a broader principle: style-sensitive benchmarks often become most informative when they evaluate the property that matters to human observers in the final output domain.

Source: https://www.emergentmind.com/topics/stylebench-h