MegaStyle-FLUX: Diffusion Model for Artistic Transfer

• MegaStyle-FLUX is a diffusion model that integrates transformer-based multi-modal attention for style-conditional text-to-image synthesis.
• It leverages a large, curated MegaStyle-1.4M dataset and a style-supervised contrastive encoder to achieve high intra-style consistency and inter-style diversity.
• Experimental evaluations demonstrate state-of-the-art performance in style retrieval and transfer, driving scalable advances in artistic style synthesis.

MegaStyle-FLUX is a state-of-the-art, style-conditional text-to-image diffusion model for generalizable artistic style transfer, integrating paired supervision, a scalable style dataset, and a transformer-based diffusion architecture. Developed within the MegaStyle framework, MegaStyle-FLUX extends the FLUX MM-DiT (Multi-Modal Diffusion Transformer) for disentangled, fine-grained control of style and content, leveraging a large, highly curated dataset and a style-supervised contrastive encoder to enable high intra-style consistency and inter-style diversity in stylized image synthesis (Gao et al., 9 Apr 2026).

1. Architectural Foundations

MegaStyle-FLUX is instantiated as a paired-supervised, multi-modal diffusion model built atop the FLUX MM-DiT backbone. The architecture employs a frozen VAE encoder/decoder for compression to a four-fold down-sampled latent space. Both the reference style image ($x_\mathrm{ref}$) and the target content image ($x_\mathrm{tgt}$) are encoded into patch-level tokens via this VAE. Textual conditioning is supplied through a pretrained text encoder, yielding tokenized content prompts. The multi-modal DiT then jointly attends over style tokens ($Z_\mathrm{style}\in\mathbb{R}^{N_s\times d}$), noisy target tokens ($Z_\mathrm{noisy}\in\mathbb{R}^{N_x\times d}$), and text tokens ($T\in\mathbb{R}^{N_t\times d_t}$). Style tokens receive a shifted Rotary Position Embedding (RoPE) to avoid positional collision with content tokens. During training, only the DiT's transformer weights (including LoRA adapters, rank 128) are updated; VAE, text encoder, and style encoder remain static.

Key Architectural Components

Component	Role	Training Status
VAE encoder/decoder (FLUX)	Latent representation and reconstruction	Frozen
Pretrained text encoder	Content prompt tokenization	Frozen
MegaStyle Encoder (SigLIP-SoViT)	Style discrimination and similarity evaluation	Trained separately
MM-DiT backbone (LoRA-augmented)	Joint multi-modal self- and cross-attention	Trainable

2. Diffusion Process and Training Mechanisms

MegaStyle-FLUX utilizes the standard denoising score-matching paradigm in the latent space, distinct from the rectified flow (RF) paradigm of baseline FLUX, to facilitate paired image transfer. For each instance, the target content image ($x_\mathrm{tgt}$) is encoded as $z_0$. At a randomly sampled timestep $t$ with DDPM noise schedule $\{\alpha_t\}$, $$z_t = \sqrt{\alpha_t}\, z_0 + \sqrt{1-\alpha_t}\,\epsilon$$, where $x_\mathrm{tgt}$0. The model predicts the noise $x_\mathrm{tgt}$1 conditioned on ($x_\mathrm{tgt}$2, $x_\mathrm{tgt}$3, $x_\mathrm{tgt}$4, $x_\mathrm{tgt}$5), and the training loss is:

$x_\mathrm{tgt}$6

as opposed to the velocity field regression of rectified flow in “vanilla” FLUX. No auxiliary adversarial, perceptual, or separate style/content losses are applied during backbone training. Flux MM-DiT self- and cross-attention layers enable direct fusion of style, content, and text embeddings.

3. MegaStyle Encoder and Style Supervision

A dedicated style encoder, the MegaStyle-Encoder, derived from SigLIP's SoViT backbone (400M parameters, 384px, patch-14), is trained for robust, discriminative style embeddings using a Style-Supervised Contrastive Learning (SSCL) objective. The encoder's output is L2-normalized and—after training—serves as an evaluation tool for style retrieval and as a metric of style similarity.

Two terms comprise the SSCL loss:

• Intra-style supervised contrastive term:

$x_\mathrm{tgt}$7

($x_\mathrm{tgt}$8 is the normalized embedding of image $x_\mathrm{tgt}$9, $Z_\mathrm{style}\in\mathbb{R}^{N_s\times d}$0 the set of positive samples, $Z_\mathrm{style}\in\mathbb{R}^{N_s\times d}$1 all negatives, temperature $Z_\mathrm{style}\in\mathbb{R}^{N_s\times d}$2).

• Inter-modal image-text contrastive term:

$Z_\mathrm{style}\in\mathbb{R}^{N_s\times d}$3

The combined SSCL loss is $Z_\mathrm{style}\in\mathbb{R}^{N_s\times d}$4.

4. MegaStyle-1.4M Dataset and Data Curation

The MegaStyle-1.4M dataset underpins MegaStyle-FLUX. It consists of 1.4 million style-consistent, content-diverse image pairs. The dataset is constructed as follows:

• Source pools: 2M "style" images (JourneyDB, WikiArt, stylized LAION) and 2M general LAION images.
• Prompt extraction: Captioning with Qwen3-VL, deduplication via Nemo-Curator, resulting in 1M unique style and 1M content prompts.
• Clustering and balancing: Hierarchical k-means and cap sampling yield 170K balanced fine-grained style prompts and 400K content prompts.
• Style pairing: Each style prompt is paired with $Z_\mathrm{style}\in\mathbb{R}^{N_s\times d}$5 different content prompts to ensure intra-style consistency and inter-style diversity (Gao et al., 9 Apr 2026).

Ablation results show significant gains in style transfer metrics when using MegaStyle-1.4M compared to smaller, noisier sets (JourneyDB, OmniStyle-150K).

5. Training Procedures and Hyper-parameters

Two core components are trained independently:

• MegaStyle-Encoder: Trained with SSCL for 30 epochs, batch size 8192, AdamW (lr=5e−4, wd=0.01), $Z_\mathrm{style}\in\mathbb{R}^{N_s\times d}$6.
• MegaStyle-FLUX: Trained for 30,000 steps, batch size 8, AdamW (lr=1e−4), input resolution 512×512 (VAE latents), LoRA rank 128. The FlowMatchScheduler is used with 40 inference steps, CFG scale of 4.0. No staging, curriculum, or two-phase training is reported; all pairs are used from the beginning (Gao et al., 9 Apr 2026).

The FLUX backbone is initialized from the public FLUX.1-dev checkpoint, keeping VAE and text encoder weights fixed.

6. Experimental Results and Evaluation Metrics

Evaluation demonstrates that MegaStyle-FLUX achieves state-of-the-art style transfer and retrieval on several benchmarks:

• Style Retrieval: The MegaStyle-Encoder markedly outperforms CLIP, CSD, and SigLIP across multiple datasets (StyleRetrieval mAP@1 for MegaStyle-Encoder (SoViT): 88.46 vs. CLIP (ViT-L): 9.29).
• Style Transfer: On the main benchmark, MegaStyle-FLUX records Style 76.16, Text 23.20, Human Style 31.37, Human Text 28.72, surpassing alternative methods including StyleShot, StyleAligned, and Attn-Distill.
• Ablation: Style score and text alignment are highest when trained on MegaStyle-1.4M (Style 76.16/Text 23.20), compared to JourneyDB (34.56/21.12) and OmniStyle-150K (51.49/23.02).
• Qualitative findings: MegaStyle-FLUX excels in reproducing brushwork, texture, and color across diverse styles. Noted failure modes are primarily due to association bias in style prompt captions, e.g., cultural motifs overrepresented by Qwen-Image, rather than architectural defects (Gao et al., 9 Apr 2026).

7. Insights and Limitations

Empirical ablations confirm:

• Intra-style consistency is crucial. Models trained on inconsistent style-content pairs (e.g., JourneyDB) fail at robust stylization.
• Inter-style diversity is necessary for generalization to unseen styles; MegaStyle-1.4M’s 170K distinct prompts enable this, whereas coarser sets (OmniStyle-150K) limit the model to simple color transfer.
• Style encoder robustness persists across synthetic, real-art, and unseen data.
• The role of specific architectural modifications (e.g., shifted RoPE, LoRA rank) is not explicitly ablated.
• Fine-tuning legacy style transfer models (e.g., StyleShot-FLUX) on MegaStyle-1.4M yields improvements but does not achieve parity with MegaStyle-FLUX.

A plausible implication is that the modular integration of large-scale, high-consistency datasets with multi-modal diffusion transformers offers a reproducible route to scalable generalization in style transfer, contingent on the availability of massive prompt-annotated data and robust style encoders.

---

MegaStyle-FLUX synthesizes recent advances in transformer-based diffusion modeling, dataset curation, and large-scale contrastive pretraining to achieve high-fidelity, generalizable style transfer. Its design, combining paired supervision, modular multi-modal attention, and strongly curated data, distinguishes it within the style transfer literature (Gao et al., 9 Apr 2026).