FFE-Bench: Fine-Grained Facial Expression Benchmark

• FFE-Bench is an evaluation framework for fine-grained facial expression editing that uses continuous 12-dimensional affective annotations.
• It measures system performance through four metrics: structural confusion, editing accuracy, linear controllability, and the identity-preservation vs expression-editing trade-off.
• The benchmark leverages the Flex Facial Expression dataset with same-identity multi-expression pairs from real-world and anime imagery to enable nuanced affect control.

FFE-Bench is the evaluation framework introduced with PixelSmile for fine-grained facial expression editing. In this usage, “FFE” denotes “Flex Facial Expression,” and the benchmark is built around the Flex Facial Expression dataset, which provides same-identity multi-expression pairs with continuous 12-dimensional affective annotations across real-world and anime domains. FFE-Bench evaluates four complementary properties of expression editing systems—structural confusion, editing accuracy, linear controllability, and the identity-preservation versus expression-editing trade-off—and thereby shifts evaluation from discrete emotion labels toward continuous affective control (Hua et al., 26 Mar 2026).

1. Definition and conceptual scope

FFE-Bench was formulated to address a specific failure mode in fine-grained facial expression editing: intrinsic semantic overlap among adjacent emotions such as fear and surprise, or anger and disgust. The benchmark is paired with the Flex Facial Expression dataset and is intended to measure whether an editing system can separate nearby expression semantics while preserving identity and supporting continuous intensity control (Hua et al., 26 Mar 2026).

The benchmark’s stated role is to evaluate four aspects crucial for controllable expression editing: structural confusion, editing accuracy, linear controllability, and the trade-off between expression editing and identity preservation. This framing is important because it does not treat expression editing as a purely categorical generation problem. Instead, it evaluates whether edits are distinguishable, target-consistent, monotonic under continuous control, and compatible with identity fidelity. In the PixelSmile formulation, these properties are assessed under both real-world and anime imagery, which places the benchmark across two distinct visual domains rather than restricting it to a single portrait distribution (Hua et al., 26 Mar 2026).

A central design choice is the replacement of one-hot labels with continuous affect. The underlying dataset uses a 12-dimensional continuous score vector, which is intended to represent semantic overlap and to capture fine boundaries on the affective manifold. This suggests that FFE-Bench is not only an evaluation suite for output correctness, but also a measurement framework for disentanglement under graded affective supervision.

2. Dataset foundation: the Flex Facial Expression corpus

FFE-Bench is grounded in the Flex Facial Expression dataset, which contains 60,000 images, about 30,000 per domain for real and anime data. The dataset was constructed to provide same-identity variation across 12 expressions with continuous intensities for each expression category. The paper presents this as resolving two typical limitations of prior datasets: a lack of paired expressions for the same identity and reliance on one-hot labels (Hua et al., 26 Mar 2026).

The dataset follows a four-stage collect–compose–generate–annotate pipeline. In the real domain, the base identity collection uses approximately 6,000 portraits curated from the Human Images Dataset and Matting Human Datasets, after automated face detection and manual verification. These images include close-up and full-body shots and vary in pose, lighting, and background context. In the anime domain, the source pool spans 207 productions covering 629 characters, with approximately 6,000 high-quality character images retained after quality filtering and detection. The paper describes this domain as introducing stylistic diversity, including 2D, CG, manga, and sketch styles (Hua et al., 26 Mar 2026).

The expression taxonomy consists of six basic emotions—happy, sad, angry, fear, surprise, disgust—and six extended emotions—confused, contempt, confident, shy, sleepy, anxious. Expression prompt composition is organized through a structured prompt library decomposed into facial attribute components such as mouth, eyebrows, and eyes, and validated with a vision-LLM to remove anatomically inconsistent or semantically conflicting descriptions. Controlled expression generation is then performed with Nano Banana Pro using dual-part prompts combining a global category with localized facial attributes (Hua et al., 26 Mar 2026).

Each image is annotated with a 12-dimensional continuous score vector

$\mathbf{v} \in [0,1]^{12}.$

These scores are estimated by Gemini 3 Pro. A subset of samples is verified by human annotators, and ambiguous or low-confidence samples are removed through consistency checks and manual spot verification. The paper states that inter-rater reliability metrics are not reported; it notes verified subsets and consistency filtering but does not provide numerical reliability statistics (Hua et al., 26 Mar 2026).

The paper does not define fixed train, validation, or test splits for FFE. In the reported experiments, PixelSmile trains LoRA adapters using FFE-style triplets and evaluates on benchmark tasks defined by FFE-Bench across six basic and twelve extended expressions. The Hugging Face page hosts the benchmark with assets and scripts for reproducible evaluation (Hua et al., 26 Mar 2026).

3. Metrics and measurement pipeline

FFE-Bench uses four primary metrics. All expression classifications and intensity scores used in evaluation are predicted by Gemini 3 Pro, while identity similarity is computed as the average cosine similarity across ArcFace, AdaFace, and FaceNet embeddings (Hua et al., 26 Mar 2026).

Structural confusion is quantified through directed and bidirectional confusion rates:

$$C_{i \rightarrow j} = \frac{1}{N_i} \sum_{k=1}^{N_i} \mathbf{1}\bigl(\hat{y}_k^{(i)} = j\bigr), \qquad \mathrm{BCR}(i,j) = \frac{1}{2}\left(C_{i \rightarrow j} + C_{j \rightarrow i}\right).$$

The mean Structural Confusion Rate averages bidirectional confusion over a predefined set of confusing pairs:

$$\mathrm{mSCR} = \frac{1}{|\mathcal{P}|} \sum_{(i,j)\in \mathcal{P}} \mathrm{BCR}(i,j).$$

Here, the dominant expression is the argmax index of the 12-dimensional score vector. Lower mSCR indicates less off-diagonal confusion and therefore better disentanglement (Hua et al., 26 Mar 2026).

Editing accuracy is defined categorically as

$$\mathrm{Acc} = \frac{1}{N} \sum_{k=1}^{N} \mathbf{1}\bigl(\hat{y}_k = y_k^{\mathrm{tgt}}\bigr).$$

The benchmark reports Acc-6 over the six basic expressions and Acc-12 over all twelve expressions. The measurement pipeline generates one edit per target instruction, scores it with Gemini 3 Pro, takes the argmax category, and computes the success proportion (Hua et al., 26 Mar 2026).

Linear controllability is measured through Pearson correlation between the control coefficient $\alpha$ and the predicted target-expression intensity:

$$\mathrm{CLS} = \rho_{\alpha,s} = \frac{\mathrm{cov}(\alpha, s)}{\sigma_{\alpha}\,\sigma_{s}} = \frac{\sum_{t} (\alpha_t - \bar{\alpha})(s_t - \bar{s})} {\sqrt{\sum_{t}(\alpha_t - \bar{\alpha})^2}\,\sqrt{\sum_{t}(s_t - \bar{s})^2}}.$$

Uniformly spaced $\alpha_t \in [0,\alpha_{\max}]$ are sampled along the interpolation path, edits are generated for each $\alpha_t$, and Gemini 3 Pro supplies the target-expression intensity $s_t$. CLS-6 and CLS-12 are averaged over six and twelve expressions respectively. Higher CLS indicates more monotonic and more linear control (Hua et al., 26 Mar 2026).

The identity-preservation versus expression-editing trade-off is summarized by the harmonic mean

$$\mathrm{HES} = \frac{2\,S_E\,S_{\mathrm{ID}}}{S_E + S_{\mathrm{ID}}},$$

where $$C_{i \rightarrow j} = \frac{1}{N_i} \sum_{k=1}^{N_i} \mathbf{1}\bigl(\hat{y}_k^{(i)} = j\bigr), \qquad \mathrm{BCR}(i,j) = \frac{1}{2}\left(C_{i \rightarrow j} + C_{j \rightarrow i}\right).$$0 is the VLM-based target expression score and $$C_{i \rightarrow j} = \frac{1}{N_i} \sum_{k=1}^{N_i} \mathbf{1}\bigl(\hat{y}_k^{(i)} = j\bigr), \qquad \mathrm{BCR}(i,j) = \frac{1}{2}\left(C_{i \rightarrow j} + C_{j \rightarrow i}\right).$$1 is the identity similarity computed from ArcFace, AdaFace, and FaceNet. High HES requires both strong target expression intensity and preserved identity (Hua et al., 26 Mar 2026).

Taken together, these metrics partition the problem into disentanglement, categorical target fidelity, continuous control quality, and identity consistency. A plausible implication is that FFE-Bench is intended to make failure modes separable rather than collapsing all outcomes into a single score.

4. Evaluation protocol and methodological context

The FFE-Bench protocol generates edits from neutral or source images using curated target-expression prompts. For continuous control, textual latent interpolation uses $$C_{i \rightarrow j} = \frac{1}{N_i} \sum_{k=1}^{N_i} \mathbf{1}\bigl(\hat{y}_k^{(i)} = j\bigr), \qquad \mathrm{BCR}(i,j) = \frac{1}{2}\left(C_{i \rightarrow j} + C_{j \rightarrow i}\right).$$2, with $$C_{i \rightarrow j} = \frac{1}{N_i} \sum_{k=1}^{N_i} \mathbf{1}\bigl(\hat{y}_k^{(i)} = j\bigr), \qquad \mathrm{BCR}(i,j) = \frac{1}{2}\left(C_{i \rightarrow j} + C_{j \rightarrow i}\right).$$3 corresponding to neutral, $$C_{i \rightarrow j} = \frac{1}{N_i} \sum_{k=1}^{N_i} \mathbf{1}\bigl(\hat{y}_k^{(i)} = j\bigr), \qquad \mathrm{BCR}(i,j) = \frac{1}{2}\left(C_{i \rightarrow j} + C_{j \rightarrow i}\right).$$4 to full target, and $$C_{i \rightarrow j} = \frac{1}{N_i} \sum_{k=1}^{N_i} \mathbf{1}\bigl(\hat{y}_k^{(i)} = j\bigr), \qquad \mathrm{BCR}(i,j) = \frac{1}{2}\left(C_{i \rightarrow j} + C_{j \rightarrow i}\right).$$5 permitted for stronger intensity. The benchmark emphasizes confusing expression pairs for disentanglement tests and aggregates results over the four metrics described above (Hua et al., 26 Mar 2026).

The benchmark is evaluated against two classes of baselines. General editing baselines include Seedream, Nano Banana Pro, GPT-Image, Qwen-Edit, FLUX-Klein, and LongCat. Continuous-control baselines include SAEdit, ConceptSlider, AttributeControl, Kontinuous-Kontext (K-Slider), and SliderEdit. The benchmark assets and scripts are hosted on Hugging Face, while the project page, code repository, model, and demo are released separately (Hua et al., 26 Mar 2026).

FFE-Bench is tightly coupled to PixelSmile’s technical design. PixelSmile uses textual latent interpolation

$$C_{i \rightarrow j} = \frac{1}{N_i} \sum_{k=1}^{N_i} \mathbf{1}\bigl(\hat{y}_k^{(i)} = j\bigr), \qquad \mathrm{BCR}(i,j) = \frac{1}{2}\left(C_{i \rightarrow j} + C_{j \rightarrow i}\right).$$6

with frozen neutral and target text embeddings. During LoRA fine-tuning, the method aligns continuous intensity through a flow-matching loss, and it uses a fully symmetric joint training procedure with a contrastive objective over confusing expression pairs. Identity preservation is supervised with a frozen ArcFace encoder, and the overall objective balances intensity alignment, contrastive disentanglement, and identity fidelity (Hua et al., 26 Mar 2026).

This methodological coupling matters because FFE-Bench is not merely a post hoc reporting suite. Its metrics are directly reflected in the training objectives of the method introduced alongside it. That correspondence is explicit in the paper’s presentation of mSCR, CLS, and HES.

5. Reported empirical behavior

On the general editing evaluation, PixelSmile achieves mSCR 0.0550, Acc-6 0.8627, Acc-12 0.6000, and ID Sim 0.6522. In the same comparison, GPT-Image records mSCR 0.1107, Acc-12 0.6300, and ID Sim 0.5056; Nano Banana Pro records Acc-6 0.8431 and ID Sim 0.7107 but a higher mSCR of 0.1754; Seedream records ID Sim 0.7221 but mSCR 0.3725; and FLUX-Klein records ID Sim 0.4146 (Hua et al., 26 Mar 2026).

On the continuous-control evaluation, PixelSmile reports CLS-6 0.8078, CLS-12 0.7305, HES 0.4723, and ID Sim 0.6522. SliderEdit reports CLS-6 0.5599, CLS-12 0.5217, and ID Sim 0.7414, but the paper notes that identity drops strongly at higher intensities. K-Slider shows negative CLS scores and unstable responses, while maintaining high average ID Sim 0.7974 largely due to weak edits. ConceptSlider and AttributeControl are described as limited in scope, with CLS-2 on happy and surprised, and weaker editing and identity consistency than PixelSmile zero-shot interpolation on Qwen-Edit (Hua et al., 26 Mar 2026).

The ablation studies further specify how FFE-Bench responds to architectural changes. Removing identity loss increases CLS but causes identity drift; removing contrastive loss collapses editing and increases confusion; removing the symmetric framework undermines disentanglement. Among the triplet formulations tested, InfoNCE provides the best balance of mSCR, CLS, and HES. Training on MEAD, which uses discrete intensities, underperforms training on FFE across metrics. This suggests that the benchmark is sensitive to continuous annotation quality and to same-identity expression diversity rather than only to raw generation capacity (Hua et al., 26 Mar 2026).

FFE-Bench does not define a formal blending metric. The paper instead presents smoothness evidence quantitatively through CLS and qualitatively through interpolation sequences and compound-expression examples, noting that 9 out of 15 basic-expression pairs yield plausible compound expressions while some pairs collapse or conflict physiologically (Hua et al., 26 Mar 2026).

6. Limitations, interpretive cautions, and naming ambiguity

The benchmark inherits several limitations from its annotation and scoring pipeline. Its evaluation depends on Gemini 3 Pro for expression scores and dominant-category prediction, and the paper notes potential bias, particularly in extended expression categories where reliability is lower. Human verification is used only on a subset, and numerical inter-rater reliability statistics are not reported. The paper also does not report confidence intervals or statistical tests, and seeds and per-method sampling details are not explicitly provided (Hua et al., 26 Mar 2026).

Dataset bias is also acknowledged. The real-portrait subset reflects typical Internet portrait distributions, including demographic imbalance such as many young adults, while the anime domain introduces ambiguous attribute labels under real-world affective categories. Some expression pairs remain intrinsically overlapping, and blending can collapse or produce conflicts, including fear plus surprise and angry plus happy (Hua et al., 26 Mar 2026).

The name “FFE-Bench” is not globally unique in the arXiv-adjacent benchmark landscape. In computer vision, it denotes the Flex Facial Expression benchmark described above (Hua et al., 26 Mar 2026). However, nearby literature uses similar strings for unrelated tasks. “FEABench” evaluates LLMs on multiphysics reasoning through COMSOL Multiphysics and finite element analysis (Mudur et al., 8 Apr 2025). “FFE-Hallu” is presented as a benchmark for figurative hallucination in Persian idioms and proverbs and is described as an “FFE-Bench” for fixed figurative expressions (Hosseini et al., 27 Jan 2026). A separate repository-level code-generation benchmark is officially named “FEA-Bench,” with the source explicitly noting that “FFE-Bench” is likely a typo for “FEA-Bench” (Li et al., 9 Mar 2025).

In that context, FFE-Bench most precisely refers to the PixelSmile benchmark when the subject is fine-grained facial expression editing. Its distinguishing characteristics are the use of same-identity multi-expression data, continuous 12-dimensional annotations, and a four-part evaluation protocol centered on disentanglement, target accuracy, controllability, and identity preservation (Hua et al., 26 Mar 2026).