SAIL-VL2: Open Multimodal Vision-Language Model

• SAIL-VL2 is an open-suite vision-language foundation model that unifies a pre-trained vision encoder, lightweight adapter, and LLM backbone with efficient sparse Mixture-of-Experts architecture.
• It employs a progressive multi-stage training regime and curated data curation to enhance fine-grained perception, visual question answering, and multimodal reasoning across images, text, and video.
• Benchmark results demonstrate that both 2B and 8B variants achieve state-of-the-art performance on over 100 benchmarks, setting a new reference for future multimodal research.

SAIL-VL2 is an open-suite vision-language foundation model (LVM) optimized for comprehensive multimodal understanding and reasoning across images, text, and video. As the direct successor to SAIL-VL, SAIL-VL2 introduces methodological, architectural, and training innovations that enable state-of-the-art performance at 2B and 8B parameter scales over 100+ benchmarks, spanning fine-grained perception tasks to advanced multimodal reasoning. Its design incorporates large-scale data curation, a progressive training paradigm, and efficient sparse Mixture-of-Experts (MoE) architectures, and is positioned as an extensible platform for open multimodal research and applications (Yin et al., 17 Sep 2025).

1. Model Architecture and Structural Innovations

SAIL-VL2 unifies three principal components: a pre-trained vision encoder (SAIL-ViT), a lightweight vision-language adapter, and a LLM backbone. The SAIL-ViT encoder, an evolved Vision Transformer, is trained to produce visual embeddings that align with the LLM's token space. SAIL-ViT is available in fixed-resolution and “AnyRes” variants (the latter using interpolation-based positional encoding for arbitrary input sizes). A two-layer MLP adapter transposes SAIL-ViT outputs into the LLM input space, enabling end-to-end multimodal processing.

A defining architectural advancement is the integration of sparse Mixture-of-Experts (MoE) modules within the LLM, as exemplified in the Qwen3-MoE variant. In these MoE designs, only a subset of expert networks is activated per input token, controlled via a gating function with auxiliary load-balancing loss. This achieves substantial parameter scaling and capacity increases without proportional computational overhead, allowing SAIL-VL2 to realize strong performance at higher scales while maintaining tractability. The architecture thus supports both classic dense LLM and efficient sparse MoE backends, resulting in a model capable of complex visual-language reasoning at scale.

2. Data Curation and Pretraining Pipeline

SAIL-VL2 employs a large-scale, quality-optimized data pipeline targeting diverse multimodal tasks including captioning, OCR, QA, and video understanding. Scoring and filtering heuristics are applied to maximize both data quality and representative coverage across task domains. Data collectors source from high-coverage corpus repositories, after which images and videos are annotated, filtered, and scored according to modality-specific criteria.

Progressive data resampling is used at both the dataset and linguistic level to maintain domain balance and linguistic variety, addressing repetitive patterning issues in open corpus pretraining. These procedures result in a highly curated, large-scale training set that supports model robustness and generalizability in downstream multimodal tasks.

3. Progressive Training Regime

Model training follows a multi-stage progressive framework:

(a) SAIL-ViT Encoder Training

• Warm-up adaptation: The adapter is tuned on 8M simple multimodal examples (image captioning, OCR) with both the vision encoder and LLM frozen, establishing a coarse modality alignment.
• Fine-grained alignment: Adapter and vision encoder are unlocked, expanding training to more complex data types (document structure, video-caption pairs) for deeper alignment.
• World knowledge injection: All parameters are unfrozen and jointly trained on a large-scale, multimodal data mix (captioning, OCR, QA, math, text), yielding a vision encoder with strong generalization.

(b) Multimodal Pretraining

Two main sub-stages:

• Basic pre-training: SAIL-ViT and LLM are pre-trained while the MLP adapter is initialized and optimized using AdaLRS, a dynamic learning rate schedule based on local loss slope:

$$\eta_{t+k} = \begin{cases} \alpha' \cdot \eta_t & \text{if } v(\alpha' \cdot \eta_t) > v(\eta_t) + 2\epsilon \ \beta' \cdot \eta_t & \text{if } v(\alpha' \cdot \eta_t) < v(\eta_t) - 2\epsilon \ \eta_t & \text{otherwise} \end{cases}$$

where $v(\cdot)$ estimates the loss slope over a sliding window, $\epsilon$ is an estimation error bound, and $\alpha', \beta'$ are scaling coefficients.

• Multi-task pre-training: General caption/OCR data, synthetic VQA, math, and QA datasets are integrated using two-phase resampling to promote both inter- and intra-dataset diversity.

(c) Post-Training: Thinking-Fusion SFT–RL Hybrid

Supervised fine-tuning (SFT) is conducted with incrementally increasing data/task difficulty, focusing on vision, text, and video-image mixtures. Long chain-of-thought (LongCoT) SFT explicitly trains the model to generate stepwise reasoning before final answers (using tags and \verb|\boxed{}| LaTeX-style formatting). Reinforcement learning (RL) with variants of Proximal Policy Optimization (PPO) further tunes answer correctness and formatting. The final “Think-Fusion” SFT phase allows the model to dynamically choose between stepwise explanations or concise answers, balancing interpretability and efficiency.

4. Benchmark Results and Empirical Performance

SAIL-VL2 has been benchmarked on 106 datasets encompassing fine-grained perception (OCR, document layout, chart understanding), general VQA, multimodal reasoning, video understanding, and mathematical reasoning (e.g., MMMU, MathVista). Key results include:

• Base model achieves competitive or state-of-the-art results on challenging multi-step reasoning tasks (e.g., MMMU, MathVista), with strong “out-of-the-box” performance across visual domains.
• Dense and MoE variants at 2B and 8B scales score at the top among open-source models—on the OpenCompass leaderboard, SAIL-VL2-2B ranks first for open-access models under 4B parameters.
• Scalability of the MoE backend allows parameter count to increase with modest impact on inference/training cost, translating to higher available capacity for complex multimodal tasks.

This performance portfolio substantiates the claim that data curation, architectural scaling by sparse experts, and an explicit reasoning-centric SFT-RL paradigm can together yield high-quality, efficient multimodal models.

5. Applications and Broader Impact

SAIL-VL2’s multimodal reasoning and perception abilities support diverse applications:

• Document and Chart Processing: Advanced OCR modules enable structured extraction from high-resolution document layouts and complex chart images.
• Visual Question Answering: Chain-of-thought mechanisms facilitate transparent Q&A in educational, scientific, and technical support contexts.
• Video Analytics: Pretraining on video data with balanced image-video finetuning supports video search, summarization, and moderation.
• Interactive Multimodal Systems: As an extensible open-suite model, SAIL-VL2 can be embedded in virtual assistants, robotics, or real-time perception-action loops that demand robust multimodal understanding.

Implications for research include the demonstration that compact, progressively trained models with MoE scaling achieve high task coverage and efficiency. SAIL-VL2’s open-source status establishes it as a reference baseline for the next wave of multimodal AI research and development, and its SFT–RL “thinking-fusion” approach provides a template for further advances in model interpretability and adaptive reasoning.

6. Methodological Directions and Research Implications

The design of SAIL-VL2 suggests several methodological lessons for multimodal foundation models:

• Progressive data curation and multi-stage training regimes maximize both alignment and generalization without incurring the inefficiencies associated with brute-force data or parameter scaling.
• MoE architectures with dynamic, load-balanced expert selection deliver scalable capacity while avoiding the intractability of fully dense scaling.
• Hybrid instruction-tuning and reinforcement learning with explicit task-oriented reasoning steps improve both the transparency and reliability of model outputs.
• Open-suite extensibility, where models are designed for integration into broader research infrastructure, can accelerate the advancement of community-driven multimodal AI systems.

A plausible implication is that future multimodal agents for scientific, educational, and general AI domains will rely on variants of the SAIL-VL2 paradigm, favoring progressive, quality-focused training with modular, sparsely activated architectures, and explicit reasoning pipelines over undifferentiated model scaling.