MegaStyle-Encoder: Vision Transformer for Style

• MegaStyle-Encoder is a vision transformer–based style encoder trained via style-supervised contrastive learning on the MegaStyle-1.4M dataset.
• It uses a fine-tuned SigLIP backbone to achieve high intra-style consistency and inter-style diversity for reliable image retrieval.
• Integrated in MegaStyle-FLUX pipelines, it sets new benchmarks in style transfer and similarity measurement tasks.

MegaStyle-Encoder is a vision transformer–based style encoder trained via style-supervised contrastive learning on the MegaStyle-1.4M dataset for expressive and reliable extraction of style-specific image representations. Developed as part of the MegaStyle framework, it is optimized for high intra-style consistency, inter-style diversity, and effective style retrieval and similarity measurement. The encoder, built upon a fine-tuned SigLIP backbone, achieves state-of-the-art performance on the StyleRetrieval benchmark and serves as a critical component in style transfer pipelines, notably with the FLUX-based diffusion transformer MegaStyle-FLUX (Gao et al., 9 Apr 2026).

1. Design and Architecture

MegaStyle-Encoder employs a SigLIP vision transformer backbone, either "siglip-so400m-patch14-384" (SoViT, $D=384$) or ViT-Large ($D=1{,}024$), to embed images into a style-centric feature space. Unlike generalist encoders, it is fine-tuned explicitly for style discrimination rather than content or class semantics. Style embeddings $\mathbf{z}\in\mathbb{R}^D$ are L2-normalized; no further projection or transformation heads are used beyond this direct output.

The encoder is primarily trained in a style-supervised contrastive setting, taking synthetic images generated under controlled style and content conditions. Each image–style prompt pair is used both as an image anchor and through prompt embedding, facilitating robust intra-style and inter-modal contrast supervision.

2. Dataset Foundations and Curation

The MegaStyle-Encoder’s performance and expressivity are enabled by the MegaStyle-1.4M dataset. This resource contains:

• 170,000 distinct, fine-grained style prompts, each covering unique textually described artistic styles.
• 400,000 diverse content prompts, textually describing the semantic content of an image.
• 1.4 million image–style pairs, generated combinatorially to ensure balanced intra-style consistency and inter-style diversity.

Data curation relies on large source image pools: JourneyDB (1M), WikiArt (80K), and LAION-Aesthetics (1M). Prompts are derived via Qwen3-VL using specific instruction templates: 32-word for style, 64-word for content. Hierarchical k-means sampling and deduplication yield a balanced and non-redundant prompt gallery.

3. Training Objectives and Contrastive Losses

MegaStyle-Encoder optimization leverages a combined style-supervised contrastive learning (SSCL) objective, integrating two loss types:

• Intra-modal Supervised Contrastive Loss (SCL):

$$\mathcal{L}_{\mathrm{scl}} = \frac{1}{MN} \sum_{i=1}^{MN} \left( -\frac{1}{|\mathcal{P}(i)|} \sum_{p\in\mathcal{P}(i)} \log \frac{\exp(\mathbf{z}_i^\top \mathbf{z}_p/\tau)}{\sum_{a\in\mathcal{A}(i)}\exp(\mathbf{z}_i^\top \mathbf{z}_a/\tau)} \right)$$

Promotes high similarity between embeddings of images with the same style prompt.

• Inter-modal Image–Text Contrastive Loss (ITC):

$$\mathcal{L}_{\mathrm{itc}} = \frac{1}{MN^2} \sum_{i=1}^{MN}\sum_{j=1}^{MN} \log\left(1 + \exp\left(-y_{ij}\,\mathbf{z}_i^\top \mathbf{t}_j\right)\right)$$

Encourages alignment between image embeddings and their associated style prompt embeddings.

• Combined SSCL:

$$\mathcal{L}_{\mathrm{sscl}} = \mathcal{L}_{\mathrm{scl}} + \mathcal{L}_{\mathrm{itc}}$$

Only the encoder backbone ($\mathcal{E}_\theta$) is updated; the text encoder ($\phi$) remains fixed.

Training proceeds for 30 epochs, batch size 8,192, learning rate $5\times10^{-4}$, weight decay 0.01, and contrastive temperature $\tau=0.07$.

4. Evaluation Benchmarks and Performance

Across multiple retrieval and transfer metrics, MegaStyle-Encoder demonstrates state-of-the-art effectiveness.

Style Retrieval Benchmark

When evaluated on 2,400 held-out styles (4 gallery images/style, 4 queries), results are as follows (mAP@k and Recall@k):

Method	mAP@1	mAP@10	Recall@1	Recall@10
CLIP (ViT-L)	9.29	6.46	9.29	31.56
CSD (ViT-L)	45.60	37.78	45.60	79.18
MegaStyle-Encoder (ViT-L)	87.26	85.98	87.26	97.61
SigLIP (SoViT)	10.43	7.83	10.43	36.32
MegaStyle-Encoder (SoViT)	88.46	86.77	88.46	97.66

These results confirm that MegaStyle-Encoder dramatically outperforms general vision-LLMs in style retrieval (Gao et al., 9 Apr 2026).

Stylization and Style Similarity

MegaStyle-Encoder style space is used to measure style alignment between reference and generated images, serving as a target metric for generative style transfer models. It is also employed in human preference studies for style perception and ranking.

5. Integration within MegaStyle-FLUX and the Broader Ecosystem

MegaStyle-Encoder is core to the MegaStyle-FLUX pipeline, where it provides style embeddings injected into a DiT-based diffusion transformer for stylized image synthesis:

• Reference style image $D=1{,}024$0 is processed via the (frozen) VAE encoder to produce style tokens $D=1{,}024$1.
• These style tokens are injected with shifted RoPE into the MM-DiT backbone alongside noisy image latents and text tokens.
• High-fidelity, style-aligned outputs are generated that reflect both the target content caption and the style as quantified by MegaStyle-Encoder.

Its learned embedding structure is demonstrated to effectively guide style transfer with superior style alignment, image quality, and content preservation compared to prior approaches.

6. Ablation Studies and the Role of Data Quality

Ablations show the necessity of intra-style consistency and inter-style diversity for both learning expressive encoder features and for downstream style transfer:

• Training FLUX on datasets with poor intra-style consistency (JourneyDB) or limited inter-style diversity (OmniStyle-150K) yields significant reductions in style alignment metrics (34.56 vs. 76.16 for style, when using JourneyDB vs. MegaStyle-1.4M).
• Ablations on injection mechanisms confirm that both the dataset and the encoder architecture are required for best performance. StyleShot-FLUX, for example, achieves only 57.06 style alignment, versus 76.16 for MegaStyle-FLUX using MegaStyle-Encoder and MegaStyle-1.4M.

This suggests that high-quality, prompt-curated, and properly balanced datasets play an essential role in enabling strong, generalizable style representations.

7. Significance and Future Impact

MegaStyle-Encoder advances the field by facilitating reliable, large-scale, and fine-grained style similarity measurement. Its performance in retrieval and transfer demonstrates its utility not only for generative pipelines but also for tasks such as style-conditioned retrieval, clustering, ablation-driven analysis, and benchmarking of style transfer models. A plausible implication is that the principles of contrastive learning on prompt-combinatorial, high-consistency datasets may generalize to other attribute-centric representation learning beyond style.

MegaStyle-Encoder sets a new technical standard for style sensitivity and alignment in neural image encoders, supported by a transparent and scalable data curation methodology (Gao et al., 9 Apr 2026).