---
title: 'EvoMakeup: Unified Diffusion Makeup Editing'
url: https://www.emergentmind.com/topics/evomakeup
type: topic
---

# EvoMakeup: Unified Diffusion Makeup Editing

EvoMakeup is a unified, diffusion-based framework for high-fidelity and controllable facial makeup editing introduced together with MakeupQuad, a large-scale quadruplet dataset designed to disentangle identity and makeup supervision [2508.05994]. In its canonical formulation, EvoMakeup supports full-face reference-based editing, partial-region reference-based editing, and text-driven makeup editing within a single model, while aiming to preserve the input face’s identity, pose, and background. The framework is notable for treating data structure as the central technical bottleneck: it is built around quadruplets $\{I, R, M, D\}$ in which the source image $I$ and edited result $M$ share identity, pose, and background, while the reference image $R$ and edited result $M$ share identical makeup, and $D$ provides a textual makeup description [2508.05994].

## 1. Problem formulation and conceptual basis

Facial makeup editing requires modifying an input face so that its identity, pose, and background are preserved while its makeup appearance matches a given specification. In the EvoMakeup formulation, that specification may be provided by a reference image or by text. The central technical difficulty is the trade-off between identity preservation and makeup fidelity: models that strongly match the reference makeup often drift in identity, whereas models that preserve identity often weaken the transferred makeup [2508.05994].

EvoMakeup addresses this problem by insisting on structured paired supervision rather than conventional loosely paired or unpaired data. The framework explicitly assumes that effective training needs data in which the source image $I$ and the edited result $M$ share the same identity, pose, and background, and the reference image $R$ and the edited result $M$ share identical makeup. This quadruplet structure is intended to make identity and makeup independently learnable, and to support precise regional control such as eyes, lips, eyebrows, nose contour, and decorative stickers [2508.05994].

A potential source of confusion is terminological. In the 2025 paper titled “EvoMakeup,” the term denotes a specific unified diffusion framework built on OmniGen and trained with MakeupQuad [2508.05994]. Related work later describes an “EvoMakeup” system as well-suited for evolutionary search over transparent makeup mask parameters, but that usage is presented as an extension and relation rather than as the original definition of the named framework [2509.02445]. This suggests that, in current usage, EvoMakeup functions both as the name of a concrete model and as a broader design direction for controllable makeup systems.

## 2. MakeupQuad and the quadruplet data model

MakeupQuad is the data substrate on which EvoMakeup is built. Each sample contains a non-makeup face $I$, a makeup reference image $R$ of a different identity but with the same makeup as the result, an edited result $M$ that shares identity with $I$ and makeup with $R$, and a detailed textual description $D$ of the makeup [2508.05994]. The dataset contains approximately 10,000 quadruplets, with 10,000 non-makeup faces, 10,000 makeup reference images, 10,000 edited results, and 10,000 textual descriptions. Its image resolution is greater than 512 pixels on the short side, and its coverage includes daily or light makeup, heavy makeup, and special-effect makeup, as well as region-specific categories such as eye, lip, eyebrow, nose contour, and decorative stickers [2508.05994].

The generation pipeline is explicitly synthetic and multi-stage. Non-makeup faces $I$ are synthesized with FLUX from GPT-4V-generated prompts. Makeup faces $M$ are created by applying SeedEdit to $I$ with approximately 1,000 diverse GPT-4V makeup prompts spanning natural to special-effect styles. Resulting samples are filtered by facial alignment, identity similarity, and realism, and then blended into the original background via facial masks so that edits remain confined to the face. Makeup references $R$ are then created through a two-step identity-disentangling procedure: LivePortrait first animates $M$ to obtain $M'$, and MimicBrush transfers $M'$ onto a different non-makeup face $I'$, producing reference candidates that preserve makeup while changing identity. Text descriptions $D$ are refined by GPT-4V so that the descriptions match the makeup attributes actually present in $M$ [2508.05994].

The dataset also contains partial-region data. Masks for eyes, eyebrows, lips, nose, and stickers are obtained via Mediapipe and Grounded SAM. A full-face model trained on coarse quadruplets is used to synthesize partial edits, GPT-4V filters those outputs for consistency, fidelity, and precision, and small curated regional sets are used to fine-tune LoRA adapters per region. Those adapters are then used to generate large-scale coarse partial quadruplets for eyes, lips, eyebrows, nose, and stickers, again with GPT-4V text annotations [2508.05994].

Quality control is a central part of the dataset design. InsightFace-based similarity checks enforce identity preservation between $I$ and $M$ and identity separation between $R$ and $I/M$. Pose and background consistency are preserved through alignment constraints and mask-based blending. GPT-4V performs sample scoring for makeup similarity and non-edited region preservation, and manual selection is applied to top-ranked partial-region samples [2508.05994]. The paper does not specify an internal train, validation, and test split for MakeupQuad; instead, external benchmarks are used for evaluation.

## 3. Model architecture and co-evolution training

EvoMakeup is built on OmniGen, described as a diffusion model with a latent-diffusion-style backbone. The VAE is kept fixed, while the remaining parameters are fine-tuned [2508.05994]. Conditioning is unified across modalities and includes image reference conditioning through learned encoders and cross-attention, regional masks $m$ for partial edits, and text descriptions $D$ through OmniGen’s text encoding pathway. Partial-region specialization is handled by LoRA adapters during data synthesis, but the final EvoMakeup model is unified and does not require per-region adapters at inference [2508.05994].

Its training pipeline is explicitly multi-stage and is described as co-evolution or multi-stage distillation. Stage 0 fine-tunes OmniGen, except for the VAE, on coarse full-face quadruplets for 20K steps with AdamW, learning rate $1 \times 10^{-4}$, batch size 8, on 8 NVIDIA A800 GPUs. The resulting full-face transfer model is then used to create and filter partial triples. Region-specific LoRA adapters of rank 8 are trained for 1K steps per region at learning rate $3 \times 10^{-4}$ to expand the partial dataset. The unified model is then fine-tuned in two co-evolution rounds, each lasting 50K steps, with repeated pseudo-label generation, filtering, and rebalancing [2508.05994].

A central design problem in this loop is image degradation across rounds, including blur, color shifts, and structural drift. EvoMakeup addresses that with a dynamic data composition strategy anchored by task reversal—specifically makeup removal—and undistilled non-makeup faces used as ground truth anchors. According to the paper, this stabilizes training and preserves identity and realism during iterative refinement [2508.05994].

Notably, the paper does not provide explicit loss formulas. It does, however, describe the framework as a conditional diffusion fine-tuning regime whose practical goals are identity preservation, makeup fidelity, and non-edited region preservation. This omission is itself consequential: the distinctive contribution lies less in a novel published objective and more in the combination of structured quadruplet supervision, dynamic distillation control, and unified multimodal conditioning.

## 4. Editing modes and controllability

EvoMakeup’s stated scope is multi-task makeup editing within a single model. The first mode is full-face reference transfer, in which the model receives a non-makeup face $I$ and a reference image $R$ and edits the full facial makeup while preserving identity and non-edited content. The second mode is partial transfer, in which inputs include $I$, $R$, and masks for specified regions such as eyes, lips, eyebrows, nose, or stickers. The third mode is text-driven editing, in which the model receives $I$ together with a textual description $D$ and applies the requested makeup semantics without requiring a reference face [2508.05994].

Regional controllability is built into both the data and the model. Masks derived from Mediapipe and Grounded SAM constrain edits to intended regions during training data generation, while the final diffusion model accepts regional masks as conditioning. Text-driven control is learned from the textual descriptions paired with quadruplets, and the unified setup is intended to support full-face, partial, and text-based editing without switching architectures [2508.05994].

The framework’s generalization claim is unusually strong: EvoMakeup is trained solely on synthetic data yet is reported to generalize well to real-world benchmarks. The stated rationale is that paired quadruplets disentangle the invariants—identity, pose, background—from the editable attributes—makeup—while high resolution, diverse makeup styles, mask-based blending, and GPT-4V-based filtering reduce synthetic artifacts and improve semantic alignment [2508.05994].

Inference speed is not reported. The paper describes the unified model as a standard conditional diffusion pipeline with mask support, but exact timings, memory footprints, and deployment latencies are not given [2508.05994]. A plausible implication is that EvoMakeup prioritizes edit fidelity, controllability, and data structure over real-time rendering constraints.

## 5. Empirical performance and ablation evidence

EvoMakeup is evaluated on 1,000 pairs each from MT, Wild-MT, and LADN. The reported metrics are FID for realism, identity similarity measured as FSim with InsightFace, makeup similarity measured as MSim with GPT-4V-based comparison of makeup attributes between $R$ and $M$, and Non-Edited Region Preservation measured as NEP through GPT-4V-based assessment of pose, background, and hair consistency between $I$ and $M$ [2508.05994].

On MT, EvoMakeup reports FID $=57.5$, FSim $=0.73$, MSim $=6.21$, and NEP $=9.94$. On Wild-MT, it reports FID $=63.8$, FSim $=0.66$, MSim $=6.46$, and NEP $=9.94$. On LADN, it reports FID $=32.9$, FSim $=0.65$, MSim $=6.30$, and NEP $=9.91$ [2508.05994]. The paper states that EvoMakeup achieves the best or competitive FID and clearly leads in MSim and NEP across datasets, and further argues that it is the only method to simultaneously maintain high identity and high makeup fidelity.

User studies reinforce that interpretation. In top-1 voting on MT, EvoMakeup receives 61.2 for makeup consistency, 18.7 for identity, 48.6 for NEP, and 63.6 for quality. On Wild-MT, the corresponding values are 77.2, 25.4, 42.8, and 61.1. On LADN, they are 72.0, 26.1, 37.8, and 60.2 [2508.05994]. The paper characterizes the method as consistently ranked best in makeup consistency, NEP, and overall quality.

The ablation results are particularly important because the framework is explicitly iterative. On MT, the initial system EvoMakeup$^0$ reports FSim $=0.41$, MSim $=4.53$, and NEP $=9.79$. After one co-evolution round, EvoMakeup$^1$ reaches FSim $=0.72$, MSim $=5.94$, and NEP $=9.83$. The final EvoMakeup$^2$ reaches FSim $=0.73$, MSim $=6.21$, and NEP $=9.94$ [2508.05994]. These numbers directly support the paper’s central claim that iterative data-model co-optimization improves identity preservation, makeup fidelity, and non-edited region preservation rather than degrading them.

## 6. Relation to adjacent systems and research directions

EvoMakeup sits within a broader technical landscape of makeup transfer, virtual try-on, and appearance editing. Earlier inverse-graphics work such as “Deep Graphics Encoder for Real-Time Video Makeup Synthesis from Example” maps a reference image to a 17-dimensional graphics parameter vector and renders the result per frame; its mobile profiling reports approximately 29 ms per frame on Galaxy S21 and approximately 40 ms per frame on Pixel 4 [2105.06407]. EvoMakeup differs structurally: it is not framed as parameter estimation for a non-differentiable renderer, but as unified diffusion editing trained on structured quadruplets [2508.05994].

A different line of work emphasizes decoupled rendering. “Towards High-Fidelity, Identity-Preserving Real-Time Makeup Transfer: Decoupling Style Generation” separates transparent RGBA makeup mask extraction from graphics-based mask rendering, so that once the mask is extracted, per-frame inference reduces to keypoint detection, parsing, warping, and alpha blending, with temporal consistency arising from mask reuse [2509.02445]. That system is explicitly real-time after mask extraction, whereas EvoMakeup does not report runtime. The contrast is therefore between data-centric diffusion editing and graphics-centric transparent-mask rendering.

Thematically broader makeup generation is represented by “Gorgeous: Create Your Desired Character Facial Makeup from Any Ideas,” which uses Stable Diffusion v2.1, a ControlNet-based Makeup Formatting module, textual inversion from three to five reference images of arbitrary type, and a face-focused inpainting pipeline to produce thematic makeup without requiring a face in the reference images [2404.13944]. Compared with that formulation, EvoMakeup remains centered on structured facial supervision, reference-based editing, and text-guided control derived from paired quadruplets rather than from arbitrary non-face inspirations.

These comparisons clarify a common misconception. EvoMakeup is not primarily a real-time renderer, a pure inverse-graphics estimator, or a thematic non-face-reference generator. In its named 2025 formulation, it is best understood as a unified diffusion editing framework whose main novelty lies in MakeupQuad, co-evolution training, and the resolution of the identity–makeup trade-off through structured synthetic supervision [2508.05994].

## 7. Limitations, interpretive issues, and open questions

The paper identifies several limitations. Because MakeupQuad is constructed using generative and editing models, minor deviations in makeup details or identity can persist despite filtering. Some loss of very fine skin texture remains at ultra-high resolutions above $1024 \times 1024$, and editing quality can degrade under complex lighting [2508.05994]. These are significant qualifications, because they indicate that the framework’s strongest results are obtained under benchmark settings that remain more controlled than unrestricted production imagery.

The dataset’s demographic coverage is described as varied in ethnicity and gender, but exact counts are not disclosed. The paper states that exact demographic balance is not quantified and explicitly notes future work on auditing and improving fairness across skin tones, ages, and genders [2508.05994]. Ethical concerns are also acknowledged: makeup editing can be misused for deceptive media, and the paper recommends usage guidelines, watermarking options, and consent-aware workflows.

A broader interpretive issue concerns the name itself. In adjacent literature, “EvoMakeup” is sometimes invoked as a system concept for evolutionary search over makeup mask parameters, objectives, and constraints, particularly in relation to decoupled transparent-mask pipelines [2509.02445]. This suggests a possible future convergence between the current diffusion-based EvoMakeup and optimization-driven personalization frameworks. At present, however, the established arXiv usage anchored by MakeupQuad refers to a data-centric, diffusion-based makeup editing system rather than to an optimization algorithm in the narrow sense [2508.05994].

Source: https://www.emergentmind.com/topics/evomakeup