FLUX-Makeup: Identity-Preserving Makeup Transfer

• FLUX-Makeup is a makeup-editing paradigm that transfers cosmetic styles to a source face while maintaining identity and spatial integrity.
• The framework spans multiple approaches, including diffusion-transformer, layer-based, and real-time methods, to achieve robust makeup synthesis.
• It leverages advanced feature injection and high-quality paired data to enhance makeup fidelity, identity preservation, and background consistency.

Searching arXiv for FLUX-Makeup and closely related makeup-transfer papers to ground the article in the current literature. FLUX-Makeup denotes a set of makeup-editing systems centered on transferring or synthesizing facial cosmetics while preserving facial identity, but in the cited literature the term is used in more than one technical context. The title “FLUX-Makeup: High-Fidelity, Identity-Consistent, and Robust Makeup Transfer via Diffusion Transformer” names a diffusion-transformer-based framework built on FLUX-Kontext, with the source image as native conditional input and a decoupled reference pathway for makeup information injection (Zhu et al., 7 Aug 2025). The same label also appears in descriptions of a single-reference, layer-based system with explicit illumination transfer and of a real-time virtual try-on framework based on transparent makeup mask extraction and graphics-based rendering (Jin et al., 2019); (Chau et al., 2 Sep 2025). Across these usages, the recurring objective is to reproduce reference cosmetics under strong constraints on identity consistency, spatial correctness, and robustness.

1. Scope, task definition, and terminological usage

In the diffusion-transformer formulation, makeup transfer is defined as follows: given a source image $x^{src}$ and a reference makeup image $x^{ref}$, the goal is to synthesize an output $x^{out}$ such that facial structure and identity remain close to the source, makeup appearance resembles the reference, background stays consistent with the source, and transfer works across complex real-world settings (Zhu et al., 7 Aug 2025). The paper explicitly frames prior difficulties in terms of weak supervision from unpaired data, auxiliary control modules that inject extra errors, and failure to balance fidelity and identity preservation.

Within the cited literature, the designation “FLUX-Makeup” is attached to multiple systems with distinct operating assumptions. This suggests that the term is best understood as a label spanning several method families rather than a single canonical architecture.

Paper	Formulation	Distinctive mechanism
“FLUX-Makeup: High-Fidelity, Identity-Consistent, and Robust Makeup Transfer via Diffusion Transformer” (Zhu et al., 7 Aug 2025)	source-reference makeup transfer	FLUX-Kontext, source image as native condition, RefLoRAInjector, HQMT
“Facial Makeup Transfer Combining Illumination Transfer” (Jin et al., 2019)	single-reference facial makeup transfer system	layer decomposition and illumination transfer
“Towards High-Fidelity, Identity-Preserving Real-Time Makeup Transfer: Decoupling Style Generation” (Chau et al., 2 Sep 2025)	real-time virtual makeup try-on	transparent makeup mask extraction and graphics-based mask rendering

The common denominator is not a shared backbone but a shared problem setting: transferring makeup style while preserving identity. The major divergence lies in what is treated as the primary object of transfer—layered appearance components, transparent RGBA masks, or reference-conditioned latent features.

2. Methodological lineage within makeup-transfer research

FLUX-Makeup emerges in a field already structured by correspondence-based, transformer-based, and diffusion-based formulations. “Semi-parametric Makeup Transfer via Semantic-aware Correspondence” proposes SpMT, which combines a non-parametric Semantic-aware Correspondence module with a parametric SPADE-based decoder. Its central claim is that non-parametric techniques have high potential for pose, expression, and occlusion discrepancies, and its reported results include the lowest FID across several makeup styles and the highest SSIM for identity preservation, with SSIM at $0.89$ versus $0.87$, $0.80$, and $0.81$ for BeautyGAN, PSGAN, and SCGAN (Zhu et al., 2022). In that line of work, robustness is obtained by semantic-guided patch transfer constrained by facial component masks.

“Facial Attribute Transformers for Precise and Robust Makeup Transfer” reformulates makeup transfer as structured attribute transfer. FAT uses transformer-style correspondence between source and reference faces, while Spatial FAT integrates thin plate splines so that geometric attributes can be transferred in addition to color and texture. The paper claims high-fidelity color transfer, geometric transformation of facial parts, robustness to poses and shadows, and high-resolution face generation; it also reports inference speeds of $22.0$ ms for FAT and $26.4$ ms for Spatial FAT, versus $146.6$ ms for PSGAN (Wan et al., 2021). This line emphasizes explicit semantic correspondence and deformation.

Diffusion-based work broadens the conditioning regime. “Gorgeous: Create Your Desired Character Facial Makeup from Any Ideas” does not require faces in the reference images and instead learns thematic inspiration from three to five arbitrary images through Character Settings Learning, a Makeup Formatting Module, and a Makeup Inpainting Pipeline (Sii et al., 2024). “DreamMakeup: Face Makeup Customization using Latent Diffusion Models” is training-free and supports reference images, explicit RGB colors, and textual descriptions through early-stopped DDIM inversion, pixel-space customization, and latent-space diffusion refinement (Park et al., 13 Oct 2025). “Flux-Sculptor: Text-Driven Rich-Attribute Portrait Editing through Decomposed Spatial Flow Control” is not a dedicated makeup-transfer system, but it explicitly demonstrates localized edits such as “lipstick,” “makeup,” and “rosy cheeks” using prompt-aligned mask prediction and staged flow control (He et al., 5 Jul 2025).

Against this background, the diffusion-transformer FLUX-Makeup is distinguished by removing auxiliary face-control components and directly leveraging source-reference image pairs, rather than relying on landmark-conditioned control pipelines or purely text-driven editing (Zhu et al., 7 Aug 2025).

3. Diffusion-transformer formulation: FLUX-Kontext and RefLoRAInjector

The diffusion-transformer FLUX-Makeup is built on FLUX-Kontext. Its design principle is to use the source image directly as the structural and identity anchor while injecting only makeup-specific information from the reference image through a decoupled reference pathway (Zhu et al., 7 Aug 2025). The paper contrasts three setups: FLUX fine-tuning with concatenated source and reference, FLUX-Kontext + LoRA, and FLUX-Makeup. In the first, source identity and reference style conflict; in the second, conditioning becomes too strong and often causes reference-face copying; in the third, source-native conditioning is retained while reference makeup is injected by RefLoRAInjector.

RefLoRAInjector is implemented as a plug-in module inside self-attention layers. If the reference image is encoded into features $x^{ref}$0, the reference pathway produces low-rank key and value tensors

$x^{ref}$1

with

$x^{ref}$2

These are concatenated with the original attention tensors,

$x^{ref}$3

so that reference information affects attention through a controlled side-channel rather than by naive fusion into the backbone stream.

The paper characterizes this decoupling as crucial for isolating makeup features from identity content, reducing interference with source structural conditioning, and avoiding direct face copying. The resulting framework is trained with LoRA parameters rather than full model updates, with a fixed text prompt “Makeup.”, LoRA rank $x^{ref}$4, $x^{ref}$5 iterations, the Prodigy optimizer, and learning rate $x^{ref}$6. At inference, it uses $x^{ref}$7 sampling steps and guidance scale $x^{ref}$8, and only the facial region of the reference input is used, extracted via face parsing to avoid background interference (Zhu et al., 7 Aug 2025).

4. Supervision and the HQMT paired-data pipeline

A major part of the FLUX-Makeup contribution is the generation of HQMT, a High-Quality Makeup Transfer dataset designed to provide more accurate supervision during diffusion training (Zhu et al., 7 Aug 2025). The stated motivation is that existing paired or pseudo-paired datasets are often poorly aligned, inconsistent in background, weak in makeup transformation, noisy, and not scalable.

The pipeline begins with GPT-4 generation of over $x^{ref}$9 makeup-related keywords such as “colorful” and “smoky.” Each keyword is tested by generating images using FLUX-Kontext, and prompts that distort background or produce poor styles are discarded, leaving $x^{out}$0 curated makeup descriptors. These are then applied to FFHQ, using the prompt template

$x^{out}$1

with five makeup adjectives assigned per image. This produces about $x^{out}$2 stylized outputs from roughly $x^{out}$3 FFHQ faces.

The raw pairs are then filtered by three checks: facial misalignment filtering, makeup-failed filtering, and inconsistent background filtering. Misalignment filtering uses BiSeNet face parsing for regions including eyes, teeth, and facial contour, discarding a pair when the number of non-overlapping pixels exceeds a threshold,

$x^{out}$4

Makeup-failed filtering computes per-pixel absolute intensity differences over facial regions and removes pairs with too few modified pixels relative to $x^{out}$5 and $x^{out}$6. Background inconsistency filtering performs an analogous difference test over non-face regions using $x^{out}$7 and $x^{out}$8. After filtering, about $x^{out}$9 high-quality pairs remain (Zhu et al., 7 Aug 2025).

The paper treats this data pipeline as a core technical contribution rather than an auxiliary implementation detail. That emphasis is consistent with its broader argument that diffusion-based makeup transfer is especially sensitive to supervision quality.

5. Evaluation, ablation, and reported performance

The diffusion-transformer FLUX-Makeup is evaluated on MT, Wild-MT, and LADN using CLIP-I for makeup transfer fidelity, SSIM for identity preservation and structural similarity, and L2-M for background consistency (Zhu et al., 7 Aug 2025). CLIP-I is defined as

$0.89$0

and L2-M as

$0.89$1

Dataset	FLUX-Makeup (CLIP-I / SSIM / L2-M)	Best prior diffusion baseline in the reported table
MT	0.668 / 0.879 / 5.18	Stable-Makeup: 0.649 / 0.792 / 10.13
Wild-MT	0.677 / 0.875 / 5.96	Stable-Makeup: 0.661 / 0.735 / 11.63
LADN	0.731 / 0.862 / 5.20	Stable-Makeup: 0.715 / 0.790 / 10.93

The reported interpretation is that FLUX-Makeup is best on all metrics across all datasets: highest makeup fidelity, best identity preservation, and best background consistency (Zhu et al., 7 Aug 2025). The gain is especially large in L2-M, which the paper treats as evidence of markedly improved background stability.

The ablations are equally central. Framework ablation compares FLUX + LoRA, FLUX-Kontext + LoRA, and FLUX-Makeup on LADN: $0.89$2

$0.89$3

$0.89$4

The paper emphasizes that the best numerical CLIP-I does not necessarily mean best visual quality, since FLUX-Kontext + LoRA tends to paste the reference face. Training-data ablation compares Stable-Makeup data, unfiltered HQMT, and HQMT, with pass rates $0.89$5, $0.89$6, and $0.89$7, and corresponding LADN scores $0.89$8, $0.89$9, and $0.87$0 (Zhu et al., 7 Aug 2025). These numbers are presented as direct evidence for the paper’s data-quality thesis.

6. Alternative FLUX-Makeup formulations: illumination transfer and real-time mask rendering

A distinct earlier formulation appears in “Facial Makeup Transfer Combining Illumination Transfer,” which describes FLUX-Makeup as a single-reference facial makeup transfer system developed into Windows platform application software (Jin et al., 2019). Its pipeline has five stages: whitening and smoothing, alignment, layer decomposition, layer transfer, and illumination transfer. Whitening uses OpenCV Color Balance; smoothing uses bilateral filtering in CIELAB space; alignment uses a modified Active Shape Model to find $0.87$1 landmark points and affine transformation to warp the reference face. The face is then decomposed into a facial structure layer, a facial color layer, and a facial detail layer. Detail is copied directly,

$0.87$2

color is alpha-blended on skin pixels with $0.87$3,

$0.87$4

and structure is modified through an explicit illumination-transfer rule with $0.87$5. The paper’s main claim is that black, dark, and black-and-white facial makeup can be effectively transferred by introducing illumination transfer, and it reports operation “within seconds,” including about $0.87$6 seconds on an iPhone 6 for a $0.87$7 image pair.

A later framework described in the cited data as FLUX-Makeup addresses real-time virtual makeup try-on by decoupling transparent makeup mask extraction from graphics-based mask rendering (Chau et al., 2 Sep 2025). Here the transferable object is a transparent RGBA makeup mask rather than a fully rendered stylized face. Separate U-Net-based extraction models are trained for eye, lip, and cheek makeup using pseudo-ground-truth masks from a graphics-based rendering pipeline and an unsupervised K-means branch for eye makeup. The method introduces alpha-weighted reconstruction, explicit alpha loss, and a lip color loss based on a frozen color regressor. Its real-time deployment consists of extracting the transparent mask once, then reusing and warping it across video frames via lightweight face alignment and parsing. Reported quantitative results include, on LADN, FID $0.87$8, LPIPS $0.87$9, PSNR $0.80$0, and FID(I) $0.80$1; and on Wild, FID $0.80$2, LPIPS $0.80$3, PSNR $0.80$4, and FID(I) $0.80$5 (Chau et al., 2 Sep 2025).

These two formulations differ fundamentally from the FLUX-Kontext-based diffusion-transformer system. The former treats illumination as a transferable component of makeup style; the latter treats makeup as an identity-free transparent layer reusable in real time; the diffusion-transformer version treats reference cosmetics as attention-injected features conditioned on a source-native generative backbone.

7. Position in the broader field

The diffusion-transformer FLUX-Makeup occupies a specific point in the current makeup-editing landscape. It is reference-driven rather than theme-driven, unlike Gorgeous, which learns from three to five arbitrary reference images and explicitly notes that current metrics such as FID, CSD, and DreamSim are not makeup-specific (Sii et al., 2024). It is paired-data-trained rather than training-free, unlike DreamMakeup, which supports reference images, RGB colors, and textual descriptions through early-stopped DDIM inversion and cross-attention composition (Park et al., 13 Oct 2025). It does not attempt explicit geometric transfer in the sense of Spatial FAT with thin plate splines (Wan et al., 2021), nor does it rely on prompt-aligned localization and staged rectified-flow control as in Flux-Sculptor for text-driven edits such as “lipstick” and “makeup” (He et al., 5 Jul 2025).

Several research pressures visible in adjacent work remain active around FLUX-Makeup-like systems. One is precision of correspondence and spatial localization, emphasized by semantic-aware patch transfer and transformer attention (Zhu et al., 2022); (Wan et al., 2021). A second is richer conditioning, including arbitrary thematic references and text or RGB control (Sii et al., 2024); (Park et al., 13 Oct 2025). A third is fairness-aware disentanglement of cosmetics from skin tone, made explicit in the transparent-mask framework, which argues that failures to separate semitransparent makeup from identity features raise fairness concerns (Chau et al., 2 Sep 2025). A fourth is evaluation: the lack of makeup-specific metrics is already identified as a limitation in creative diffusion-based makeup generation (Sii et al., 2024).

Within that broader trajectory, FLUX-Makeup in the diffusion-transformer sense is best characterized as a source-native, reference-injected, paired-data-trained system that seeks to eliminate auxiliary face-control components and the error accumulation they induce (Zhu et al., 7 Aug 2025). Its reported contribution is not simply higher generative quality, but a reallocation of control: identity and background are anchored by the source input, while cosmetics are injected through a lightweight reference pathway under strong paired supervision.