Kandinsky 5.0 Image Lite Diffusion Model

• Kandinsky 5.0 Image Lite is a 6-billion-parameter text-to-image diffusion model that integrates a Cross-Attention Diffusion Transformer (CrossDiT) and Flow Matching for effective latent-space synthesis.
• It employs a multi-stage training pipeline with supervised fine-tuning and RL-based post-training, achieving state-of-the-art performance with low FID scores and enhanced CLIP-scores.
• Optimized through a sophisticated data curation process and distributed training techniques, the model ensures efficient, high-quality photorealistic and artistic image generation for open-source research.

Kandinsky 5.0 Image Lite is a 6-billion-parameter foundation model for text-to-image and image-editing synthesis, developed as part of the Kandinsky 5.0 family for high-resolution visual content generation. Its architecture centers on a Cross-Attention Diffusion Transformer (CrossDiT) backbone and employs a Flow Matching training paradigm for efficient and effective learning of latent-space generative dynamics. The model distinguishes itself through architectural efficiency, extensive data-driven training protocols, and a multi-stage training pipeline culminating in RL-based post-training. Optimizations in both systems and mathematical formulation enable state-of-the-art photorealistic and artistic synthesis suitable for public open-source deployment and academic research (Arkhipkin et al., 19 Nov 2025).

1. Architectural Foundation and Components

Kandinsky 5.0 Image Lite adopts a latent-space diffusion approach, with core components including a FLUX.1-dev VAE encoder, time step embeddings, CLIP and Qwen2.5-VL text encoders, and a lightweight Linguistic Token Refiner (LTF). The denoiser operates through a stack of 50 CrossDiT blocks, integrating self-attention, cross-attention, and MLP sub-blocks. The total parameter count approximates 6 billion, with the major allocation residing in the CrossDiT backbone (≈ 5.3B trainable parameters), complemented by additional modules (LTF: 0.2B, time embedding and adapters: 0.3B).

Architectural advances include the elimination of expensive vision-text token concatenation (unlike preceding MMDiT-style designs) and the integration of Rotary Position Encodings (RoPE) for both spatial and temporal axes, facilitating compatibility with video extensions. Adaptive Normalization layers are implemented to robustly fuse multimodal embeddings.

Module Breakdown (Parameter Allocation)

Module	Parameters (B)	Notable Specs
CrossDiT Backbone (Trainable)	≈ 5.3	50 blocks, d=2560, MLP 4d
Qwen2.5-VL (Text Encoder, frozen)	7	Embedding dim=3584, len=256
CLIP ViT-L/14 (frozen)	0.2	Embedding dim=768, len=77
Linguistic Token Refiner (LTF)	0.2	2 CrossDiT-like blocks
Misc./Adapters	0.3	Time emb., norm, masks, etc.

2. Mathematical Formulation and Inference Strategy

The diffusion process is defined in the latent space via a probability flow ODE, leveraging Flow Matching to train the model to predict velocity fields of latent trajectories. Let $x_t$ denote the latent at time $t$, and $v(x_t, t; \theta)$ the instantaneous velocity prediction. Training minimizes the squared flow matching objective:

$$\mathcal{L}_{\mathrm{pretrain}} = \mathbb{E}_{t, x_0, \epsilon}\left\|v_\theta(x_t,t) - v_{\mathrm{true}}(x_t,t)\right\|_2^2,$$

where $x_t = x_0 + t \cdot \epsilon$, $\epsilon \sim \mathcal{N}(0, I)$. Inference employs classifier-free guidance with scale $s$:

$$v_{\mathrm{cfg}} = v(x_t, t \mid y) + s [v(x_t, t \mid y) - v(x_t, t \mid \varnothing)].$$

3. Data Acquisition and Preprocessing

The model is trained on approximately 500 million text-to-image (T2I) examples and 150 million image editing instruction pairs. Data sources include LAION-5B, COYO, and other public image repositories. The data curation pipeline follows a multi-stage protocol:

1. Image Quality Filters: Minimum resolution threshold (short side ≥ 256 px), deduplication (perceptual hashes), watermark detection (ResNeXt + YOLO).
2. Quality Assessment: Use of TOPIQ and Q-Align, ensuring technical and aesthetic scores.
3. Text and Complexity Filtering: CRAFT for text detection, SAM 2 + Sobel edges for complexity.
4. Annotation and Captioning: Automated object detection (YOLOv8), CLIP classification, and synthetic captioning with InternVL2-26B, InternLM3-8B, and Qwen2.5VL-32B.
5. Image Editing Pair Creation: Similarity matching via CLIP, DINO, geometric verification (LoFTR + RANSAC), and filtering for crop avoidance.
6. Instruction Generation: GLM 4.5-LoRA for linguistic instructions, curated by human evaluators.
7. SFT Subset: ≈153,000 high-quality, expert-curated image–caption pairs, filtered for technical (>4) and aesthetic (>2) Q-Align scores.

4. Training Protocol and Supervised/RL Alignment

The training pipeline comprises sequential pre-training, supervised fine-tuning (SFT), and reinforcement learning (RL)-based post-training:

• Pre-Training: Staged by resolution (LR, MR, HR), each employing specific step counts, batch sizes, learning rates, and spatial dimension schedules. The optimizer is AdamW (β₁ = 0.9, β₂ = 0.95, $\epsilon = 1\times10^{-8}$) with gradient clipping and linear warmup.
• Supervised Fine-Tuning: Weight averaging ("model soup") across 9 visual-LLM (VLM)–defined domains and their subdomains, guided by validation performance and human judgment.
• RL-based Post-Training: Qwen2.5-VL-7B reward model trained with cross-entropy on relative pairs (pre-train vs. SFT vs. real). Direct Reward Fine-Tuning (DRaFT-K) is applied to the final $K = 10$ denoising steps with the RL loss:

$$L_{\mathrm{RL}} = 1 - R(x_\nabla, x_{\mathrm{real}}, y)$$

plus a KL regularizer penalizing divergence from the SFT policy vector field:

$$\mathrm{KL}(p_{RL} \parallel p_{SFT}) = \sum_{t \in \text{last } K}\|v_{RL}(x_t, t) - v_{SFT}(x_t, t)\|^2,$$

yielding the full loss:

$$L = L_{\mathrm{RL}} + \beta_{KL} \mathrm{KL}(p_{RL} \parallel p_{SFT}), \quad \beta_{KL} = 2\times10^{-2}.$$

5. System Optimizations and Inference Efficiency

Operational enhancements facilitate both training throughput and inference efficiency:

• Data Streaming: VAE latents, pre-encoded and packaged per resolution, enable efficient loading via S3 tar archives. Dynamic batching aligns sample durations.
• Distributed Training: PyTorch FSDP+SequenceParallel sharding over 64 GPUs (CrossDiT) and 32 GPUs (text encoder), with support for NVLink islands and non-blocking checkpointing.
• Memory Efficiency: Activation checkpointing with host-RAM offload reduces peak usage by up to 40%.
• Speed Optimizations: VAE encoder accelerated by optimal tiling and torch.compile (2.5× increase); CrossDiT code refactored for torch.compile.
• Transformer and Diffusion Inference: Use of MagCache (+46% speed), FlashAttention-2 (or SageAttention) for 512², and the bypassing of NABLA optimizations in static image mode.

Single H100 GPU throughput is 13 seconds per 1024² image (100 NFEs, ≈ 0.08 img/sec), with distillation to 16 NFEs reaching ≈ 2.5 seconds per image (≈ 0.40 img/sec).

6. Empirical Performance and Evaluation

Automated and human evaluations demonstrate the empirical efficacy of Kandinsky 5.0 Image Lite:

• COCO 30k Benchmark:
  • FID (↓): HR-pretrain ≈ 9.8; after SFT ≈ 7.2; after RL-post-train ≈ 6.4
  • CLIP-Score (↑): pretrain 0.220; SFT 0.264; RL 0.276
• Human Side-by-Side (Visual Quality [VQ] / Prompt Following [PF]):

| Comparison | VQ (K5.0 Lite) | VQ (Others) | PF (K5.0 Lite) | PF (Others) | |---------------------------|---------------|-------------|---------------|-------------| | Kandinsky 5.0 vs FLUX.1 | 0.67 | 0.33 | 0.52 | 0.48 | | Kandinsky 5.0 vs Qwen-Img | 0.64 | 0.36 | 0.49 | 0.51 |

These results indicate a pronounced lead in visual quality and competitive prompt adherence.

7. Significance and Application Scope

Kandinsky 5.0 Image Lite is positioned as a high-performing, open-source generative model for both photorealistic and artistic image synthesis, leveraging comprehensive architectural and system optimizations and alignment procedures. The careful design of its data pipeline, diffusion-based mathematical foundation, and multi-domain supervised and RL alignment yield a model suitable for diverse generative and editing applications, consistent with empirical state-of-the-art across standard benchmarks and human judgment criteria (Arkhipkin et al., 19 Nov 2025).