Qwen3-VL-Embedding Multimodal Model

• Qwen3-VL-Embedding is a large-scale multimodal transformer model that unifies text, image, video, and document image representations.
• Its three-stage training—contrastive pre-training, supervised fine-tuning with a reranker, and distillation merging—optimizes retrieval accuracy and flexible dimensionality trade-offs.
• Empirical results demonstrate state-of-the-art performance on benchmarks like MMEB-V2, with strong robustness and low vulnerability under moderate visual distortions.

Qwen3-VL-Embedding is a large-scale, transformer-based multimodal embedding model developed as part of the Qwen3-VL model family. It is designed for unified representation and high-precision retrieval across heterogeneous modalities, including text, image, document images, and video. The model utilizes staged contrastive and distillation-based training, supports Matryoshka Representation Learning for flexible dimensionality trade-offs, handles long and multilingual inputs, and achieves state-of-the-art (SOTA) empirical performance on comprehensive multimodal benchmarks (Li et al., 8 Jan 2026).

1. Model Architecture

Qwen3-VL-Embedding is architecturally grounded in the Qwen3-VL multimodal transformer, supporting seamless integration of multiple input modalities via a shared stack of causal-attention transformer layers (T=28 for the 2B model, T=36 for the 8B model) (Li et al., 8 Jan 2026). Input modalities are pre-encoded as follows:

• Text: Word-piece tokenization feeds standard 1D token embeddings.
• Image/Document-image: Images are processed via Vision Transformer (ViT)-style patch embedding, linear projection per patch, and augmented with 2D positional encodings.
• Video: Individual frames are converted into patches identical to images, with additional frame-specific and spatial positional encodings.

All modality tokens are concatenated—interleaved with explicit <|im_start|> and <|im_end|> markers—and ingested as a unified sequence to the transformer. The attention mechanism remains autoregressive but is supervised via fill-in-the-blank and retrieval objectives. The output embedding for any input is the final hidden state associated with the <|endoftext|> token. Embedding dimensionality is $d=2,048$ (2B model) or $d=4,096$ (8B model), with all modalities mapped into a shared $\mathbb{R}^d$ representation space, and retrieval similarity computed via cosine metric:

$$s(z^\phi, z^d) = \cos(z^\phi, z^d) = \frac{\langle z^\phi, z^d \rangle}{\|z^\phi\| \|z^d\|}$$

This shared embedding space underpins a broad class of multimodal retrieval, question answering, and cross-modal search tasks. Qwen3-VL-Embedding also provides foundation capabilities for downstream reranking systems such as Qwen3-VL-Reranker (Li et al., 8 Jan 2026).

2. Multi-Stage Training Methodology

The Qwen3-VL-Embedding training pipeline consists of three successive stages, each focused on refining embedding quality for retrieval and transferability (Li et al., 8 Jan 2026):

Stage 1: Large-Scale Contrastive Pre-training

The first stage leverages billions of weakly supervised query–document pairs sampled from text, images, video, and document images. The objective is a generalized InfoNCE contrastive loss. For a batch of $N$ positives with $K$ hard negatives per query, the loss can be written:

$$L_{\text{retrieval}} = -\frac{1}{N} \sum_{i=1}^N \log \frac{\exp(s(q_i, d^+_i)/\tau)}{Z_i}$$

where $Z_i$ aggregates positive, negative, and cross-modal negatives—with masking to avoid false negatives. The temperature $\tau$ is learned.

Stage 2: Multi-Task Contrastive & Supervised Fine-Tuning

In this phase, the dataset is composed of a balanced mix from curated public sets, proprietary high-quality data, and synthetic pairs. The InfoNCE objective is maintained with adapted negative sampling, and Qwen3-VL-Reranker is jointly trained on mined retrieval subsets, using a pointwise cross-entropy relevance loss:

$L_{\text{rerank}} = -\log p(l|I, q, d)$

Stage 3: Distillation & Model Merging

Distillation transmits fine-grained relevance judgments from Qwen3-VL-Reranker to the embedding model. For each query, embedding scores are converted to softmax probabilities and aligned with the teacher distribution via KL divergence:

$d=4,096$0

Empirically, stage 2 can degrade performance in classification/QA; a final merging step (as in Li et al. 2024) fuses the best aspects of previous stages into the final model $d=4,096$1 (Li et al., 8 Jan 2026).

3. Matryoshka Representation Learning (MRL)

At both training and inference, Matryoshka Representation Learning enables the model to provide embeddings at multiple truncation depths without retraining. Specifically, retrieval and distillation losses are jointly computed not only on the full embedding $d=4,096$2 but also on prefix subspaces $d=4,096$3 for $d=4,096$4:

$d=4,096$5

This approach ensures that lower-dimensional projections retain semantically meaningful alignment, facilitating trade-offs between storage/throughput and performance during deployment. Empirically, shrinking from 2048 to 1024 dimensions in MSMARCO and VL3-Syn tasks costs only 1–2% relative MRR while halving storage and doubling speed; Int8 quantization introduces negligible loss, while binary quantization is substantially harmful, especially at reduced dimensions (Li et al., 8 Jan 2026).

4. Long Input and Multilingual Handling

Qwen3-VL-Embedding inherits robust long-context and multilingual support from its Qwen3-VL foundation (Bai et al., 26 Nov 2025, Li et al., 8 Jan 2026). Key properties include:

• Maximum context length: up to 32,768 tokens;
• Text: handled with standard WordPiece tokenization, documents chunked if exceeding window;
• Image/document-image: aspect ratio preserved; resized/cropped to fit 1,280-token (~1.3M pixel) maximum;
• Video: sampled at 1 FPS, up to 64 frames; total visual token budget ≤ 4,500 tokens (~9.2M pixels);
• Multilingual capability: >30 languages are supported for both text and instruction prompts, linked to broad multilingual pretraining.

These traits enable Qwen3-VL-Embedding to handle real-world retrieval scenarios involving lengthy, complex, and polyglot data streams (Li et al., 8 Jan 2026).

5. Empirical Results and Benchmarking

Comprehensive benchmarking on MMEB-V2 (78 datasets, 9 tasks over image, video, and visual document modalities) establishes Qwen3-VL-Embedding-8B as the SOTA open model as of January 2025, with an overall MMEB-V2 score of 77.8—substantially ahead of open and closed-source baselines (Li et al., 8 Jan 2026):

Model	Size	Image overall	Video overall	VisDoc overall	MMEB-V2 all
RzenEmbed (open, 8B)	8B	75.9	55.7	81.3	72.9
IFM-TTE (closed, 8B)	8B	77.9	59.2	79.5	74.1
Seed-1.6-embedding-1215	–	78.0	67.7	82.2	76.9
Qwen3-VL-Embedding-8B	8B	80.1	67.1	82.4	77.8

Task-specific results confirm robust performance on image classification (CLS 74.2), image QA (81.1), image retrieval (80.2), and graphical document understanding (GD 92.3), among others. Training-stage ablations on the 2B model show clear step-wise improvements, with s1 (multi-task) boosting retrieval by 5.5 points relative to s0, s2 (distilled) optimizing retrieval, and s3 (merged) achieving best balance across all metrics (Li et al., 8 Jan 2026).

6. Security, Text–Image Alignment, and Robustness

Qwen3-VL-Embedding yields low-dimensional embeddings for both text and images, facilitating direct quantification of cross-modal alignment and model robustness. In typographic prompt injection studies, the normalized L2 distance between text and image embeddings demonstrates a strong negative correlation (Pearson $d=4,096$6 to $d=4,096$7, $d=4,096$8) with attack success rate (ASR) across modalities and visual degradations (Balakrishnan et al., 14 Apr 2026). Moderate visual degradations (blur, noise, low contrast) raise this embedding distance and lower ASR by up to 96%. The embedding distance can thus be operationalized as a lightweight proxy for attack risk in adversarial deployments. Qwen3-VL-Embedding's alignment is resilient to moderate distortion but breaks under extreme transformation, indicating a well-calibrated trade-off between accessibility and robustness for real-world agentic systems (Balakrishnan et al., 14 Apr 2026).

7. Positioning within the Qwen3-VL Family

Qwen3-VL-Embedding extends the core Qwen3-VL architecture, which integrates multimodal inputs through token- and patch-level embeddings; deploys interleaved-MRoPE (for 3D relative positional encoding), DeepStack fusion (multi-level ViT feature injection into transformer layers), and explicit text timestamping for video grounding (Bai et al., 26 Nov 2025). These mechanisms collectively provide the backbone for Qwen3-VL-Embedding's cross-modal alignment and retrieval capacity, supporting dense and mixture-of-experts configurations for diverse compute-performance trade-offs.

Qwen3-VL-Embedding and its companion Qwen3-VL-Reranker form a modular system for open-domain multimodal search, grounded in unified transformer-based representation, scalable and robust training, Matryoshka learning, and empirical validation at the SOTA level across the multimodal retrieval spectrum (Li et al., 8 Jan 2026, Bai et al., 26 Nov 2025).