EvoMakeup: Unified Diffusion Makeup Editing

• EvoMakeup is a unified diffusion-based framework that edits facial makeup while preserving identity, pose, and background via structured quadruplet supervision.
• It leverages MakeupQuad, a dataset of around 10,000 synthetic quadruplets, to support full-face, partial-region, and text-driven editing modes.
• Iterative co-evolution training refines makeup fidelity and identity preservation, yielding competitive realism and non-edited region consistency metrics.

EvoMakeup is a unified, diffusion-based framework for high-fidelity and controllable facial makeup editing introduced together with MakeupQuad, a large-scale quadruplet dataset designed to disentangle identity and makeup supervision (Wu et al., 8 Aug 2025). In its canonical formulation, EvoMakeup supports full-face reference-based editing, partial-region reference-based editing, and text-driven makeup editing within a single model, while aiming to preserve the input face’s identity, pose, and background. The framework is notable for treating data structure as the central technical bottleneck: it is built around quadruplets $\{I, R, M, D\}$ in which the source image $I$ and edited result $M$ share identity, pose, and background, while the reference image $R$ and edited result $M$ share identical makeup, and $D$ provides a textual makeup description (Wu et al., 8 Aug 2025).

1. Problem formulation and conceptual basis

Facial makeup editing requires modifying an input face so that its identity, pose, and background are preserved while its makeup appearance matches a given specification. In the EvoMakeup formulation, that specification may be provided by a reference image or by text. The central technical difficulty is the trade-off between identity preservation and makeup fidelity: models that strongly match the reference makeup often drift in identity, whereas models that preserve identity often weaken the transferred makeup (Wu et al., 8 Aug 2025).

EvoMakeup addresses this problem by insisting on structured paired supervision rather than conventional loosely paired or unpaired data. The framework explicitly assumes that effective training needs data in which the source image $I$ and the edited result $M$ share the same identity, pose, and background, and the reference image $R$ and the edited result $M$ share identical makeup. This quadruplet structure is intended to make identity and makeup independently learnable, and to support precise regional control such as eyes, lips, eyebrows, nose contour, and decorative stickers (Wu et al., 8 Aug 2025).

A potential source of confusion is terminological. In the 2025 paper titled “EvoMakeup,” the term denotes a specific unified diffusion framework built on OmniGen and trained with MakeupQuad (Wu et al., 8 Aug 2025). Related work later describes an “EvoMakeup” system as well-suited for evolutionary search over transparent makeup mask parameters, but that usage is presented as an extension and relation rather than as the original definition of the named framework (Chau et al., 2 Sep 2025). This suggests that, in current usage, EvoMakeup functions both as the name of a concrete model and as a broader design direction for controllable makeup systems.

2. MakeupQuad and the quadruplet data model

MakeupQuad is the data substrate on which EvoMakeup is built. Each sample contains a non-makeup face $I$0, a makeup reference image $I$1 of a different identity but with the same makeup as the result, an edited result $I$2 that shares identity with $I$3 and makeup with $I$4, and a detailed textual description $I$5 of the makeup (Wu et al., 8 Aug 2025). The dataset contains approximately 10,000 quadruplets, with 10,000 non-makeup faces, 10,000 makeup reference images, 10,000 edited results, and 10,000 textual descriptions. Its image resolution is greater than 512 pixels on the short side, and its coverage includes daily or light makeup, heavy makeup, and special-effect makeup, as well as region-specific categories such as eye, lip, eyebrow, nose contour, and decorative stickers (Wu et al., 8 Aug 2025).

The generation pipeline is explicitly synthetic and multi-stage. Non-makeup faces $I$6 are synthesized with FLUX from GPT-4V-generated prompts. Makeup faces $I$7 are created by applying SeedEdit to $I$8 with approximately 1,000 diverse GPT-4V makeup prompts spanning natural to special-effect styles. Resulting samples are filtered by facial alignment, identity similarity, and realism, and then blended into the original background via facial masks so that edits remain confined to the face. Makeup references $I$9 are then created through a two-step identity-disentangling procedure: LivePortrait first animates $M$0 to obtain $M$1, and MimicBrush transfers $M$2 onto a different non-makeup face $M$3, producing reference candidates that preserve makeup while changing identity. Text descriptions $M$4 are refined by GPT-4V so that the descriptions match the makeup attributes actually present in $M$5 (Wu et al., 8 Aug 2025).

The dataset also contains partial-region data. Masks for eyes, eyebrows, lips, nose, and stickers are obtained via Mediapipe and Grounded SAM. A full-face model trained on coarse quadruplets is used to synthesize partial edits, GPT-4V filters those outputs for consistency, fidelity, and precision, and small curated regional sets are used to fine-tune LoRA adapters per region. Those adapters are then used to generate large-scale coarse partial quadruplets for eyes, lips, eyebrows, nose, and stickers, again with GPT-4V text annotations (Wu et al., 8 Aug 2025).

Quality control is a central part of the dataset design. InsightFace-based similarity checks enforce identity preservation between $M$6 and $M$7 and identity separation between $M$8 and $M$9. Pose and background consistency are preserved through alignment constraints and mask-based blending. GPT-4V performs sample scoring for makeup similarity and non-edited region preservation, and manual selection is applied to top-ranked partial-region samples (Wu et al., 8 Aug 2025). The paper does not specify an internal train, validation, and test split for MakeupQuad; instead, external benchmarks are used for evaluation.

3. Model architecture and co-evolution training

EvoMakeup is built on OmniGen, described as a diffusion model with a latent-diffusion-style backbone. The VAE is kept fixed, while the remaining parameters are fine-tuned (Wu et al., 8 Aug 2025). Conditioning is unified across modalities and includes image reference conditioning through learned encoders and cross-attention, regional masks $R$0 for partial edits, and text descriptions $R$1 through OmniGen’s text encoding pathway. Partial-region specialization is handled by LoRA adapters during data synthesis, but the final EvoMakeup model is unified and does not require per-region adapters at inference (Wu et al., 8 Aug 2025).

Its training pipeline is explicitly multi-stage and is described as co-evolution or multi-stage distillation. Stage 0 fine-tunes OmniGen, except for the VAE, on coarse full-face quadruplets for 20K steps with AdamW, learning rate $R$2, batch size 8, on 8 NVIDIA A800 GPUs. The resulting full-face transfer model is then used to create and filter partial triples. Region-specific LoRA adapters of rank 8 are trained for 1K steps per region at learning rate $R$3 to expand the partial dataset. The unified model is then fine-tuned in two co-evolution rounds, each lasting 50K steps, with repeated pseudo-label generation, filtering, and rebalancing (Wu et al., 8 Aug 2025).

A central design problem in this loop is image degradation across rounds, including blur, color shifts, and structural drift. EvoMakeup addresses that with a dynamic data composition strategy anchored by task reversal—specifically makeup removal—and undistilled non-makeup faces used as ground truth anchors. According to the paper, this stabilizes training and preserves identity and realism during iterative refinement (Wu et al., 8 Aug 2025).

Notably, the paper does not provide explicit loss formulas. It does, however, describe the framework as a conditional diffusion fine-tuning regime whose practical goals are identity preservation, makeup fidelity, and non-edited region preservation. This omission is itself consequential: the distinctive contribution lies less in a novel published objective and more in the combination of structured quadruplet supervision, dynamic distillation control, and unified multimodal conditioning.

4. Editing modes and controllability

EvoMakeup’s stated scope is multi-task makeup editing within a single model. The first mode is full-face reference transfer, in which the model receives a non-makeup face $R$4 and a reference image $R$5 and edits the full facial makeup while preserving identity and non-edited content. The second mode is partial transfer, in which inputs include $R$6, $R$7, and masks for specified regions such as eyes, lips, eyebrows, nose, or stickers. The third mode is text-driven editing, in which the model receives $R$8 together with a textual description $R$9 and applies the requested makeup semantics without requiring a reference face (Wu et al., 8 Aug 2025).

Regional controllability is built into both the data and the model. Masks derived from Mediapipe and Grounded SAM constrain edits to intended regions during training data generation, while the final diffusion model accepts regional masks as conditioning. Text-driven control is learned from the textual descriptions paired with quadruplets, and the unified setup is intended to support full-face, partial, and text-based editing without switching architectures (Wu et al., 8 Aug 2025).

The framework’s generalization claim is unusually strong: EvoMakeup is trained solely on synthetic data yet is reported to generalize well to real-world benchmarks. The stated rationale is that paired quadruplets disentangle the invariants—identity, pose, background—from the editable attributes—makeup—while high resolution, diverse makeup styles, mask-based blending, and GPT-4V-based filtering reduce synthetic artifacts and improve semantic alignment (Wu et al., 8 Aug 2025).

Inference speed is not reported. The paper describes the unified model as a standard conditional diffusion pipeline with mask support, but exact timings, memory footprints, and deployment latencies are not given (Wu et al., 8 Aug 2025). A plausible implication is that EvoMakeup prioritizes edit fidelity, controllability, and data structure over real-time rendering constraints.

5. Empirical performance and ablation evidence

EvoMakeup is evaluated on 1,000 pairs each from MT, Wild-MT, and LADN. The reported metrics are FID for realism, identity similarity measured as FSim with InsightFace, makeup similarity measured as MSim with GPT-4V-based comparison of makeup attributes between $M$0 and $M$1, and Non-Edited Region Preservation measured as NEP through GPT-4V-based assessment of pose, background, and hair consistency between $M$2 and $M$3 (Wu et al., 8 Aug 2025).

On MT, EvoMakeup reports FID $M$4, FSim $M$5, MSim $M$6, and NEP $M$7. On Wild-MT, it reports FID $M$8, FSim $M$9, MSim $D$0, and NEP $D$1. On LADN, it reports FID $D$2, FSim $D$3, MSim $D$4, and NEP $D$5 (Wu et al., 8 Aug 2025). The paper states that EvoMakeup achieves the best or competitive FID and clearly leads in MSim and NEP across datasets, and further argues that it is the only method to simultaneously maintain high identity and high makeup fidelity.

User studies reinforce that interpretation. In top-1 voting on MT, EvoMakeup receives 61.2 for makeup consistency, 18.7 for identity, 48.6 for NEP, and 63.6 for quality. On Wild-MT, the corresponding values are 77.2, 25.4, 42.8, and 61.1. On LADN, they are 72.0, 26.1, 37.8, and 60.2 (Wu et al., 8 Aug 2025). The paper characterizes the method as consistently ranked best in makeup consistency, NEP, and overall quality.

The ablation results are particularly important because the framework is explicitly iterative. On MT, the initial system EvoMakeup$D$6 reports FSim $D$7, MSim $D$8, and NEP $D$9. After one co-evolution round, EvoMakeup$I$0 reaches FSim $I$1, MSim $I$2, and NEP $I$3. The final EvoMakeup$I$4 reaches FSim $I$5, MSim $I$6, and NEP $I$7 (Wu et al., 8 Aug 2025). These numbers directly support the paper’s central claim that iterative data-model co-optimization improves identity preservation, makeup fidelity, and non-edited region preservation rather than degrading them.

6. Relation to adjacent systems and research directions

EvoMakeup sits within a broader technical landscape of makeup transfer, virtual try-on, and appearance editing. Earlier inverse-graphics work such as “Deep Graphics Encoder for Real-Time Video Makeup Synthesis from Example” maps a reference image to a 17-dimensional graphics parameter vector and renders the result per frame; its mobile profiling reports approximately 29 ms per frame on Galaxy S21 and approximately 40 ms per frame on Pixel 4 (Kips et al., 2021). EvoMakeup differs structurally: it is not framed as parameter estimation for a non-differentiable renderer, but as unified diffusion editing trained on structured quadruplets (Wu et al., 8 Aug 2025).

A different line of work emphasizes decoupled rendering. “Towards High-Fidelity, Identity-Preserving Real-Time Makeup Transfer: Decoupling Style Generation” separates transparent RGBA makeup mask extraction from graphics-based mask rendering, so that once the mask is extracted, per-frame inference reduces to keypoint detection, parsing, warping, and alpha blending, with temporal consistency arising from mask reuse (Chau et al., 2 Sep 2025). That system is explicitly real-time after mask extraction, whereas EvoMakeup does not report runtime. The contrast is therefore between data-centric diffusion editing and graphics-centric transparent-mask rendering.

Thematically broader makeup generation is represented by “Gorgeous: Create Your Desired Character Facial Makeup from Any Ideas,” which uses Stable Diffusion v2.1, a ControlNet-based Makeup Formatting module, textual inversion from three to five reference images of arbitrary type, and a face-focused inpainting pipeline to produce thematic makeup without requiring a face in the reference images (Sii et al., 2024). Compared with that formulation, EvoMakeup remains centered on structured facial supervision, reference-based editing, and text-guided control derived from paired quadruplets rather than from arbitrary non-face inspirations.

These comparisons clarify a common misconception. EvoMakeup is not primarily a real-time renderer, a pure inverse-graphics estimator, or a thematic non-face-reference generator. In its named 2025 formulation, it is best understood as a unified diffusion editing framework whose main novelty lies in MakeupQuad, co-evolution training, and the resolution of the identity–makeup trade-off through structured synthetic supervision (Wu et al., 8 Aug 2025).

7. Limitations, interpretive issues, and open questions

The paper identifies several limitations. Because MakeupQuad is constructed using generative and editing models, minor deviations in makeup details or identity can persist despite filtering. Some loss of very fine skin texture remains at ultra-high resolutions above $I$8, and editing quality can degrade under complex lighting (Wu et al., 8 Aug 2025). These are significant qualifications, because they indicate that the framework’s strongest results are obtained under benchmark settings that remain more controlled than unrestricted production imagery.

The dataset’s demographic coverage is described as varied in ethnicity and gender, but exact counts are not disclosed. The paper states that exact demographic balance is not quantified and explicitly notes future work on auditing and improving fairness across skin tones, ages, and genders (Wu et al., 8 Aug 2025). Ethical concerns are also acknowledged: makeup editing can be misused for deceptive media, and the paper recommends usage guidelines, watermarking options, and consent-aware workflows.

A broader interpretive issue concerns the name itself. In adjacent literature, “EvoMakeup” is sometimes invoked as a system concept for evolutionary search over makeup mask parameters, objectives, and constraints, particularly in relation to decoupled transparent-mask pipelines (Chau et al., 2 Sep 2025). This suggests a possible future convergence between the current diffusion-based EvoMakeup and optimization-driven personalization frameworks. At present, however, the established arXiv usage anchored by MakeupQuad refers to a data-centric, diffusion-based makeup editing system rather than to an optimization algorithm in the narrow sense (Wu et al., 8 Aug 2025).