StyleBench-H: Perception Benchmark

Updated 4 July 2026

The paper introduces StyleBench-H, a benchmark that evaluates facial identity verification under artistic stylization based on human judgments.
StyleBench-H comprises paired real and stylized portraits, using binary same–different decisions to measure perceptual identity consistency.
The benchmark assesses models across Cross-ID, Cross-Style, and Cross-Method splits, emphasizing the need for style-agnostic identity encoders.

Searching arXiv for the primary paper and closely related work on StyleBench-H and perception-grounded stylization benchmarks. arXiv search query: "StyleBench-H StyleID stylization-agnostic facial identity recognition" StyleBench-H is a human-judged benchmark for facial identity verification under stylization, introduced as one component of the StyleID framework for stylization-agnostic facial identity recognition (Yun et al., 23 Apr 2026). It is designed to measure whether an identity encoder’s similarity judgments agree with human same–different decisions when a source portrait is transformed by artistic stylization. In the StyleID formulation, StyleBench-H serves the evaluation role, while StyleBench-S is a separate supervision set derived from psychometric recognition-strength curves and used for calibration and training (Yun et al., 23 Apr 2026). The benchmark targets a specific failure mode of conventional face-recognition systems: models calibrated on natural photographs often treat texture and palette changes as identity drift, or fail to detect identity degradation caused by geometric exaggeration in stylized renderings.

1. Definition and scope

StyleBench-H is a dataset of source portrait / stylized portrait pairs used for human perception-grounded identity verification under stylization (Yun et al., 23 Apr 2026). Each datum is a pair $(I_{\mathrm{src}}, I_{\mathrm{style}})$ , where $I_{\mathrm{src}}$ is a real face photograph and $I_{\mathrm{style}}$ is a stylized rendering. Human annotators answer the question of whether the two images depict the same person. The benchmark thereby operationalizes identity consistency as a perceptual verification problem rather than as a purely feature-space similarity problem.

The benchmark was created because standard identity encoders are typically trained and thresholded on natural face photographs, and these thresholds do not transfer reliably to stylized images (Yun et al., 23 Apr 2026). This suggests that robustness to stylization is not merely a matter of embedding invariance; it also depends on whether the encoder’s score ordering aligns with human perception across varying styles and stylization strengths. A plausible implication is that StyleBench-H occupies a different evaluative regime from ordinary face-verification benchmarks: the objective is not only recognition accuracy, but recognition accuracy under systematic appearance-domain shift.

2. Benchmark composition and stylization factors

StyleBench-H covers stylized portraits generated by three controllable stylization frameworks: IP-Adapter-faceID, InstantID, and InfiniteYou (Yun et al., 23 Apr 2026). These include both diffusion-based and flow-matching-based pipelines. The benchmark spans 10 artistic styles and 7 discrete stylization strengths, with strength indexed as

$s \in \{1/7, 2/7, \ldots, 7/7\}.$

The paper also introduces a normalized strength convention in which $s=0$ corresponds to nearly original identity and $s=1$ to maximum stylization or potential identity loss (Yun et al., 23 Apr 2026). However, it explicitly notes that the same normalized strength across methods is not perceptually equivalent. Thus, $s=0.5$ under one stylization method need not produce the same level of identity retention as $s=0.5$ under another. This is a central design point: StyleBench-H is structured to expose method-dependent perceptual degradation rather than to assume a uniform stylization scale.

The source portraits are sampled from FFHQ and filtered to remove large head rotations and images containing more than one person (Yun et al., 23 Apr 2026). This filtering constrains nuisance variation so that identity-preservation failures are more directly attributable to stylization rather than pose or multi-person ambiguity.

3. Human annotation protocol

StyleBench-H is explicitly human-judged. The benchmark uses pairwise verification judgments in which participants inspect a source image and a stylized counterpart and decide whether they depict the same person (Yun et al., 23 Apr 2026). The human-study protocol includes several quality-control steps intended to suppress careless or inconsistent annotations.

The participant pool and filtering details reported for StyleBench-H are as follows.

Aspect	Value
Recruited participants	70
Valid participants after filtering	68
Mean age	29.0
Queries per participant	91
Valid responses collected	6088
Final valid datapoints	$N_H = 3551$

The participant demographics are reported as 32 male, 33 female, and 5 not disclosed (Yun et al., 23 Apr 2026). Responses faster than image loading time, responses taking longer than 100 seconds, and responses from inconsistent participants identified using a repeated final question were removed. Two participants were discarded, leaving 68 valid participants (Yun et al., 23 Apr 2026). The final benchmark retains only true-positive and true-negative pairs and is balanced between positives and negatives.

This design indicates that StyleBench-H is not merely a collection of stylized face pairs; it is a psychometrically filtered human-perception dataset. A plausible implication is that the benchmark is better interpreted as a calibrated measurement of perceptual identity retention than as a generic verification corpus.

4. Generalization splits and evaluation regime

StyleBench-H is used under three generalization settings in StyleID (Yun et al., 23 Apr 2026).

Cross-ID

The test identities are unseen. This split evaluates generalization to new people while keeping the stylization setting within the broader training distribution.

Cross-Style

This split uses IP-Adapter and evaluates on styles unseen relative to training (Yun et al., 23 Apr 2026). The objective is to test whether a model calibrated on some artistic styles can retain human-aligned identity judgments on new styles.

Cross-Method

This is the strongest distribution-shift setting. Test identities, stylization methods, and prompts/styles are all unseen, and the split includes two unseen stylization methods: MTG and Flux.2, each paired with 8 unseen styles (Yun et al., 23 Apr 2026).

These splits make StyleBench-H a benchmark not only for stylization robustness but also for out-of-distribution generalization across identity, style family, and rendering pipeline. This suggests that the benchmark is intended to probe whether an encoder has learned style-agnostic identity structure rather than method-specific calibration heuristics.

5. Metrics and benchmark objective

StyleBench-H evaluates agreement between model similarity scores and human verification judgments using standard verification metrics, with emphasis on operating-point behavior (Yun et al., 23 Apr 2026). The key metric highlighted in the main table is TPR at FPR $=10^{-2}$ . The paper also reports verification accuracy and AUROC, and the appendix includes TPR@FPR $I_{\mathrm{src}}$ 0 and TPR@FPR $I_{\mathrm{src}}$ 1.

The core evaluative question is whether a model’s similarity function remains calibrated under stylization. The paper argues that a single global threshold does not work well across stylization families, because some methods or styles induce much larger perceptual identity drift than others (Yun et al., 23 Apr 2026). This means that photo-domain thresholds can become either too strict or too permissive in stylized domains.

Unlike fully automatic stylization metrics, StyleBench-H is centered on human binary judgments. This distinguishes it from style-similarity or reconstruction-oriented measures commonly used elsewhere in stylization research. Related work on human-aligned aesthetic evaluation similarly emphasizes pairwise human preference and rank-based agreement rather than purely feature-based scoring, although in a different task domain (Jiang et al., 15 Jan 2025). StyleBench-H applies that broader philosophy to facial identity under style transformation.

6. Relation to StyleBench-S and the StyleID calibration pipeline

StyleBench-H is the evaluation benchmark in a two-part framework. StyleBench-S is the supervision set derived from psychometric recognition-strength curves obtained through controlled 2AFC experiments (Yun et al., 23 Apr 2026). The workflow described in StyleID is: build StyleBench-H, fit recognition-vs-strength curves from human responses, derive StyleBench-S from those curves, train or calibrate an identity encoder on StyleBench-S, and evaluate on StyleBench-H.

The psychometric component estimates human recognition as a function of stylization strength, method, and style: $I_{\mathrm{src}}$ 2 This formulation captures the fact that identity degradation is not uniform across artistic styles or rendering methods (Yun et al., 23 Apr 2026). StyleBench-H supplies the perceptual endpoint against which encoders are ultimately judged; StyleBench-S converts that perception into a larger training signal.

Within the StyleID model, the calibrated encoder uses a CLIP-L backbone with LoRA adapters and combines an ArcFace-style angular-margin loss, supervised contrastive loss, and regularization toward frozen CLIP features (Yun et al., 23 Apr 2026). The paper defines the total loss as

$I_{\mathrm{src}}$ 3

with $I_{\mathrm{src}}$ 4 and $I_{\mathrm{src}}$ 5 (Yun et al., 23 Apr 2026). Although these equations belong to StyleID rather than to StyleBench-H itself, they clarify the benchmark’s function: StyleBench-H is the perceptual target that motivates calibration.

7. Empirical findings and significance

StyleBench-H reveals several systematic patterns in human and model behavior under stylization (Yun et al., 23 Apr 2026). First, human identity recognition decreases as stylization strength increases. Second, the rate of this degradation differs across stylization methods and artistic styles. Third, photo-domain identity encoders such as ArcFace and AdaFace transfer better than generic encoders such as CLIP or SigLIP2, but still fail to fully align with human judgments. Fourth, perception-calibrated training improves alignment substantially.

On StyleBench-H, the calibrated StyleID model reports the following TPR values (Yun et al., 23 Apr 2026).

Split	StyleID TPR
Cross-ID	0.9020
Cross-Style	0.9255
Cross-Method	0.7444

The paper also reports that StyleID achieves the best or near-best accuracy and AUROC on these splits (Yun et al., 23 Apr 2026). In addition, the model generalizes to artist-drawn sketches on SKSF-A, with TPR 0.8891 and AUROC 0.9922 (Yun et al., 23 Apr 2026). These results suggest that human-calibrated stylization supervision transfers beyond the exact stylization pipelines used in StyleBench-H.

The benchmark also informs downstream applications. When the calibrated StyleID encoder is used as the identity loss in JoJoGAN, the paper reports better identity preservation, better style fidelity, fewer artifacts, and stronger human and GPT preference (Yun et al., 23 Apr 2026). This suggests that StyleBench-H is not only diagnostic; it also provides a practical target for improving stylization systems that must preserve identity.

8. Conceptual position and limitations

StyleBench-H is best understood as a perception-grounded verification benchmark rather than a generic stylization dataset. Its central claim is that facial identity under stylization cannot be evaluated reliably with photo-domain thresholds or with uncalibrated similarity spaces (Yun et al., 23 Apr 2026). In that sense, it shifts evaluation from “does the embedding remain stable?” to “does the embedding behave like human identity perception?”

Several limitations are either explicit or strongly implied by the benchmark design. The benchmark is specific to facial identity under stylization and does not generalize directly to non-facial content or to broader style-transfer tasks (Yun et al., 23 Apr 2026). Its stylization coverage, while diverse, is tied to the included methods, styles, and controlled strength schedule. Human judgments are binary same–different decisions, which are appropriate for verification but do not capture richer graded notions of resemblance or recognizability. The benchmark also depends on the quality and representativeness of the participant pool and the selected portrait domain.

Even with these constraints, StyleBench-H fills a distinct methodological gap. Other style-oriented benchmarks focus on webpage layout-style consistency (Lai et al., 5 Oct 2025), speaking-style control in dialogue (Zhao et al., 8 Mar 2026), style embeddings for text (Soto et al., 30 Jun 2026, Qiu et al., 21 Feb 2025), or human-aligned aesthetic reasoning for stylization (Jiang et al., 15 Jan 2025). StyleBench-H differs in that its object of evaluation is not style generation quality per se, but identity preservation judged through the lens of human perception under stylization. This suggests a broader principle: style-sensitive benchmarks often become most informative when they evaluate the property that matters to human observers in the final output domain.