---
title: 'InternVL 2.5: Open-Source MLLM Evolution'
url: https://www.emergentmind.com/topics/internvl-2-5
type: topic
---

# InternVL 2.5: Open-Source MLLM Evolution

InternVL 2.5 is an open-source multimodal large language model (MLLM) series in the InternVL family that extends InternVL 2.0 while preserving the same high-level **ViT-MLP-LLM** architecture and emphasizing coordinated scaling of **model**, **data**, and **test-time compute** rather than architectural novelty alone [2412.05271]. The series spans models from **0.9B** to **78.4B** parameters, targets image, multi-image, video, OCR, document, multilingual, reasoning, grounding, and pure-language tasks, and is presented as narrowing the gap between open-source systems and proprietary multimodal models such as **GPT-4o** and **Claude-3.5-Sonnet**. Its central headline is that **InternVL2.5-78B** is reported as the **first open-source MLLM to exceed 70% on MMMU validation**, scoring **70.1** under the paper’s protocol that takes the maximum accuracy across direct-answer and Chain-of-Thought settings [2412.05271].

## 1. Lineage and design position

InternVL 2.5 is explicitly framed as an upgraded successor to **InternVL 2.0** that **maintains its core model architecture** while introducing stronger backbones, better data curation, a more formalized training pipeline, and explicit test-time scaling [2412.05271]. It is therefore best understood as a continuation of the modular InternVL line rather than a break from it.

Within the broader family, several adjacent works clarify what is stable across InternVL generations. **Mini-InternVL** shows that compact descendants preserve the same basic **ViT–MLP–LLM** recipe, inherit the dynamic-resolution strategy associated with **InternVL 1.5**, and treat large InternVL vision encoders as sources of transferable visual knowledge [2410.16261]. **Mono-InternVL** explores a distinct, monolithic direction in which visual encoding and language decoding are integrated into a single LLM; that work is explicitly not the InternVL 2.5 line, but it reveals that the InternVL ecosystem was simultaneously investigating latency, OCR, high-resolution handling, and language-retention tradeoffs beyond the standard modular recipe [2410.08202]. **InternVL3** is then presented as the next-generation successor to InternVL 2.5, preserving the same broad **ViT-MLP-LLM** backbone while replacing the earlier post-hoc adaptation pipeline with native multimodal pre-training and adding **Variable Visual Position Encoding (V2PE)**, **Mixed Preference Optimization (MPO)**, and new infrastructure [2504.10479].

This lineage makes clear that InternVL 2.5 occupies a transitional but central position. It consolidates the family’s durable engineering principles—strong InternViT backbones, simple MLP bridging, pixel-unshuffle token reduction, dynamic high-resolution tiling, staged training, and broad multimodal scope—before the training-paradigm change introduced by InternVL3 [2412.05271].

## 2. Architecture and model lineup

InternVL 2.5 retains the standard **“ViT-MLP-LLM”** design used in earlier InternVL releases: a vision encoder, a language model, and a **randomly initialized 2-layer MLP projector** that maps visual features into the LLM token space [2412.05271]. The architecture does not introduce a new fusion mechanism. Its reported gains are attributed instead to upgraded backbones, stronger data, and revised training and inference procedures.

A key implementation detail is **pixel unshuffle**. After the ViT processes each image tile, pixel unshuffle reduces the visual token count by **4×**: a **448×448** tile initially yields **1024 visual tokens** and is reduced to **256 visual tokens** [2412.05271]. This token compression is central to InternVL-style high-resolution processing and persists into later family members such as InternVL3 [2504.10479].

InternVL 2.5 employs dynamic high-resolution processing with a fixed tile size of **448 × 448**. The strategy extends beyond single-image inputs to **multi-image** and **video** settings. With the common threshold of **48 tiles**, images can reach **12,288 visual tokens**. For video, no tiling is used; each frame is a single **448×448** image, so **32 frames** correspond to **8,192 visual tokens** and **64 frames** correspond to **16,384 visual tokens** [2412.05271].

The family uses explicit multimodal markup. Visual content is enclosed by **`<img>`** and **`</img>`**; multi-image samples use tags such as **`Image-1`**; video frames use tags such as **`Frame-1`**; and training data is formatted in a **ChatML-style** conversation format [2412.05271].

The released lineup is as follows.

| Model | Total Params | Backbone configuration |
|---|---:|---|
| InternVL2.5-1B | 0.9B | InternViT-300M-448px-V2.5 + Qwen2.5-0.5B-Instruct |
| InternVL2.5-2B | 2.2B | InternViT-300M-448px-V2.5 + internlm2_5-1_8b-chat |
| InternVL2.5-4B | 3.7B | InternViT-300M-448px-V2.5 + Qwen2.5-3B-Instruct |
| InternVL2.5-8B | 8.1B | InternViT-300M-448px-V2.5 + internlm2_5-7b-chat |
| InternVL2.5-26B | 25.5B | InternViT-6B-448px-V2.5 + internlm2_5-20b-chat |
| InternVL2.5-38B | 38.4B | InternViT-6B-448px-V2.5 + Qwen2.5-32B-Instruct |
| InternVL2.5-78B | 78.4B | InternViT-6B-448px-V2.5 + Qwen2.5-72B-Instruct |

The two vision encoders reflect an explicit scaling strategy. **InternViT-6B-448px-V2.5** has **5.5B parameters**, **45 layers**, hidden size **3200**, **25 heads**, uses **QK-Norm** and **RMSNorm**, and is trained with **NTP** after earlier CLIP-era pretraining lineage. **InternViT-300M-448px-V2.5** has **0.3B parameters**, **24 layers**, hidden size **1024**, **16 heads**, uses standard **LayerNorm**, and is described as being distilled from **InternViT-6B-V1.5** before incremental improvement with NTP [2412.05271].

The dynamic tiling policy is formalized through aspect-ratio-aware tile selection. The paper gives candidate aspect ratios as
$$
\mathcal{R} = \left\{i / j \mid 1 \leq i, j \leq n, \, i \times j \in [n_{\min}, n_{\max}] \right\},
$$
with the target grid selected by minimizing the difference between the original image aspect ratio and candidate tiling ratios. The resized dimensions are then multiples of the tile size \(S=448\), and for multi-image samples the tile budget is divided approximately evenly across images [2412.05271]. This same tile-based reasoning is visible in the autonomous-driving adaptation of **InternVL-1.5**, where six camera views are resized, concatenated into a **2688×896** canvas, and decomposed into **twelve 448×448 sub-images plus a 448×448 thumbnail** [2412.07247].

## 3. Training pipeline, scaling strategy, and data organization

InternVL 2.5 uses a **three-stage training pipeline** [2412.05271]:

1. **Stage 1: MLP warmup**  
2. **Stage 1.5: ViT incremental learning**  
3. **Stage 2: Full-model instruction tuning**

In **Stage 1**, only the **MLP projector** is trainable; both the vision encoder and LLM are frozen. The objective is to align vision features to the LLM token space using a pre-training data mixture formatted as ChatML-style conversations and optimized with **next-token prediction (NTP)**. Dynamic high resolution is already used at this stage despite the additional cost [2412.05271].

In **Stage 1.5**, the **ViT and MLP** are trainable while the LLM remains frozen. The stage uses the same pretraining data mixture and NTP loss, but with a lower learning rate intended to avoid catastrophic forgetting and to improve the vision encoder in underrepresented domains such as **multilingual OCR**, **mathematical charts**, and other rare visual domains. The paper emphasizes that this stage need only be executed once for a given vision encoder and can then be reused when scaling to larger LLMs [2412.05271].

In **Stage 2**, the **full model** is instruction-tuned on high-quality data. The paper stresses strict data filtering, noting that even a few thousand bad samples can damage behavior, especially for CoT and long-form generation [2412.05271].

A major practical contribution is the **progressive scaling strategy**. The basic idea is to optimize InternViT with a smaller LLM during Stage 1.5, then reuse that optimized ViT with larger LLMs and skip Stage 1.5 at larger scales. This is presented as a mechanism for lowering training cost. The paper contrasts **Qwen2-VL**, reported as processing **1.4T tokens**, with **InternVL2.5-78B**, which is stated to use only about **120B tokens** while benefiting from a substantially larger **6B** vision encoder [2412.05271].

The training hyperparameters differ by size, but several settings are global across the series: **context length = 16384**, and **image tile threshold = 48** [2412.05271]. Examples include:

- **1B Stage 1**: batch **512**, LR **2e-4**, tokens **~191B**
- **1B Stage 2**: batch **512**, LR **4e-5**, epochs **4**, tokens **~176B**
- **8B Stage 1.5**: batch **1024**, LR **1e-5**, tokens **~76B**
- **78B Stage 1**: batch **512**, LR **2e-4**, tokens **~76B**
- **78B Stage 2**: batch **512**, LR **2e-5**, epochs **1**, tokens **~44B**

For larger models, weight decay is generally **0.05**, and ViT drop path in Stage 2 is often **0.4** [2412.05271].

The paper also introduces a unified loss reweighting scheme for response-length normalization. It presents the weighting as
$$
\mathcal{L} = \frac{w_i}{\sum_j w_j} \cdot \mathcal{L}_i, \quad
w_i = \begin{cases}
\frac{1}{x^0}, & \text{for token averaging} \\
\frac{1}{x^1}, & \text{for sample averaging},
\end{cases}
$$
and then states that InternVL 2.5 uses **square averaging** with
$$
w_i = \frac{1}{x^{0.5}}.
$$
The stated rationale is to avoid the bias of token averaging toward longer responses and the bias of sample averaging toward shorter responses [2412.05271]. This formulation is later inherited explicitly by InternVL3 [2504.10479].

Data organization is another defining feature. Fine-tuning data scale grows from **5.1M samples** in **InternVL 1.5** to **7.3M** in **InternVL 2.0** and **16.3M** in **InternVL 2.5** [2412.05271]. The InternVL 2.5 fine-tuning token composition is reported as:

- **single-image:** **45.92%**
- **multi-image:** **9.37%**
- **video:** **39.79%**
- **text-only:** **4.92%**

The pretraining mixture is conversation-format instruction data spanning captioning, general QA, mathematics, chart understanding, OCR, knowledge, grounding, document understanding, conversation, medical, GUI, video captioning, and video QA. Named sources include **COCO-Caption**, **OpenImages-Caption**, **Objects365-Caption**, **LAION-ZH/EN/COCO**, **ShareGPT4V**, **GQA**, **OKVQA**, **A-OKVQA**, **MAVIS**, **GeomVerse**, **ChartQA**, **PlotQA**, **FigureQA**, **SynthDoG-EN/ZH/RU/JP/KO**, **TextVQA**, **InfoVQA**, **Objects365**, **GRIT**, **RefCOCO**, **DocReason25K**, **DocVQA**, **ALLaVA**, **SVIT**, **PMC-VQA**, **Screen2Words**, **WebSight**, **Mementos**, **ShareGPT4Video**, **VideoChat2-IT**, **EgoTaskQA**, and others [2412.05271].

The paper is unusually explicit about **data filtering**. For pure text, it uses **LLM-based quality scoring** from **0 to 10**, repetition detection, heuristic rule-based filtering, and manual checking. For multimodal data, it uses repetition detection, heuristic filtering, and manual review, while exempting some high-quality academic datasets from aggressive filtering. The paper directly links this filtering pipeline to improved CoT behavior and reduced repetitive or deadlocked generations relative to InternVL 2.0 [2412.05271].

## 4. Test-time scaling and evaluation methodology

InternVL 2.5 makes **test-time scaling** a first-class design axis. The paper’s central claim is not only that better training improves performance, but also that difficult multimodal reasoning benefits from additional inference-time computation, especially through **Chain-of-Thought prompting** and **majority voting / self-consistency** [2412.05271].

The most prominent example is **MMMU validation**. For **InternVL2.5-78B**, the paper implies a direct-response MMMU score of **66.4**, and states that CoT raises it to **70.1**, a gain of **+3.7 points** [2412.05271]. The reported evaluation protocol takes the **maximum** of direct-answer and CoT performance. For an earlier model, **InternVL2-Llama3-76B**, the paper reports improvement on MMMU from **62.7** to **65.3** with CoT plus majority voting [2412.05271].

The paper argues that many open-source MLLMs underperform with CoT because of repetitive or degenerate output behavior, whereas InternVL 2.5 benefits from CoT because stricter data filtering reduced such failure modes [2412.05271]. This suggests that test-time scaling in InternVL 2.5 is not merely a prompting trick, but a design choice coupled to data curation.

The evaluation configuration also depends on modality. For images, tile limits and aspect-ratio-aware tiling control the visual context budget. For multi-image or high-resolution document settings, the paper states that **\(n_{\max}=24\)** or **36** is used; for many low-resolution image datasets, **\(n_{\max}=6\)** or **12**; for video datasets, **\(n_{\max}=1\)** [2412.05271]. Video frame sampling in training was increased from **4–24 frames** in InternVL 2.0 to **8–32 frames** in InternVL 2.5. During evaluation, the paper reports sweeps over **16**, **32**, **48**, and **64** frames on several video benchmarks and uses benchmark-specific settings such as **16 frames for MVBench** and **32 frames for CG-Bench** [2412.05271].

Later developments in the family sharpen the meaning of InternVL 2.5’s test-time strategy. InternVL3 adopts **Best-of-N** evaluation with **VisualPRM-8B** rather than relying only on CoT and majority voting, and explicitly treats this as a successor recipe rather than a restatement of InternVL 2.5 practice [2504.10479]. A plausible implication is that InternVL 2.5 represents the stage at which the InternVL line established test-time scaling as a durable principle, even though the later implementation became more specialized.

## 5. Benchmark profile and reported capabilities

InternVL 2.5 is evaluated across multimodal reasoning, mathematics, OCR, chart and document understanding, multi-image understanding, real-world perception, hallucination benchmarks, visual grounding, multilingual multimodal evaluation, video understanding, and pure-language tasks [2412.05271]. Its strongest public headline is the **MMMU** result: **InternVL2.5-78B = 70.1**, compared with **GPT-4o-20240513 = 69.1**, **Claude-3.5-Sonnet = 68.3**, **Qwen2-VL-72B = 64.5**, and **InternVL2-Llama3-76B = 62.7** under the paper’s reporting protocol [2412.05271].

On other reasoning and math-style evaluations, **InternVL2.5-78B** is reported at **61.8** on **MMMU test**, **48.6** on **MMMU-Pro overall**, **72.3** on **MathVista**, **34.9 / 32.2** on **MATH-Vision** *(testmini / full)*, and **51.7** on **MathVerse** [2412.05271]. On **OlympiadBench**, however, the result remains comparatively modest: **11.6** for **InternVL2.5-78B**, versus **25.9** for **GPT-4o** [2412.05271]. This is one of the clearest places where the paper itself presents remaining distance from top proprietary systems.

On OCR-, chart-, and document-heavy evaluations, the model is consistently strong. **InternVL2.5-78B** scores **89.1 / 95.7** on **AI2D** *(mask / no-mask)*, **88.3** on **ChartQA**, **83.4** on **TextVQA**, **95.1** on **DocVQA**, **84.1** on **InfoVQA**, and **854** on **OCRBench** [2412.05271]. It exceeds **GPT-4o** on **AI2D**, **TextVQA**, **DocVQA**, and **InfoVQA**, but trails **Qwen2-VL-72B** on **TextVQA**, **DocVQA**, **InfoVQA**, and **OCRBench** [2412.05271]. The paper explicitly acknowledges the **MMVet** family as a remaining gap even when aggregate multimodal performance is high.

On **multi-image understanding**, **InternVL2.5-78B** is reported at **63.8** on **BLINK**, **77.0** on **Mantis-Eval**, **55.8** on **MMIU**, **63.5** on **MuirBench**, **70.8** on **MMT-Bench**, and **61.1** on **MIRB** [2412.05271]. The paper notes that it surpasses **GPT-4o** on **MMIU** (**55.8 vs 55.7**) and **MMT-Bench** (**70.8 vs 65.4**), while trailing on **BLINK** and **MuirBench** [2412.05271].

On **real-world perception and robustness**, **InternVL2.5-78B** reaches **78.7** on **RealWorldQA**, **62.9** on **MME-RealWorld**, **71.4** on **WildVision**, and **77.2** on **R-Bench** [2412.05271]. The paper interprets the lower **WildVision** score relative to **GPT-4o** (**80.6**) as evidence that longer, preference-sensitive outputs remain a weakness [2412.05271].

On broader multimodal benchmarks, the same model scores **2494.5** on **MME**, **88.3 / 88.5** on **MMBench**, **87.4** on **MMBench-Video**, **72.3** on **MMVet**, **65.5** on **MMVet v2**, and **69.5** on **MMStar** [2412.05271]. It is reported as beating **GPT-4o** on **MME**, **MMBench**, and **MMStar**, while trailing on **MMVet v2** and trailing **Qwen2-VL-72B** on **MMVet** and **MMVet v2** [2412.05271].

On **hallucination-related** evaluations, **InternVL2.5-78B** is reported at **57.4** on **HallusionBench**, **3.89** on **MMHal**, **78.8** on **CRPE**, and **90.8** on **POPE** [2412.05271]. These numbers are described as competitive and improved over InternVL 2.0, but the paper explicitly states that hallucinations remain, especially in longer responses [2412.05271].

On **visual grounding**, InternVL 2.5 is one of the family’s clearest strengths. **InternVL2.5-78B** reports **93.7 / 95.6 / 92.5** on **RefCOCO**, **90.4 / 94.7 / 86.9** on **RefCOCO+**, **92.7 / 92.2** on **RefCOCOg**, for an average of **92.3** [2412.05271]. The paper calls this state of the art among the compared MLLMs, above **Qwen2-VL-72B** average **91.1** and **InternVL2-Llama3-76B** average **90.0** [2412.05271].

On **multilingual multimodal evaluation**, **InternVL2.5-78B** scores **86.3** *(en)*, **85.6** *(zh)*, **85.1** *(pt)*, **84.8** *(ar)*, **83.1** *(tr)*, and **85.4** *(ru)* on **MMMB**; **90.0** *(en)*, **89.7** *(zh)*, **87.4** *(pt)*, **83.3** *(ar)*, **84.9** *(tr)*, and **86.3** *(ru)* on **Multilingual MMBench**; and **31.9** average on **MTVQA** [2412.05271]. The authors conclude that multilingual ability appears to be largely inherited from the underlying LLM, since both InternVL2.5-78B and Qwen2-VL-72B use **Qwen 2.5** and perform similarly [2412.05271].

On **video understanding**, **InternVL2.5-78B** reaches **72.1 / 74.0** on **Video-MME** *(without / with subtitle)*, **76.4** on **MVBench**, **1.97** on **MMBench-Video**, **75.7** on **MLVU**, **63.6** on **LongVideoBench**, and **42.2 / 58.5** on **CG-Bench v1.1** [2412.05271]. A key finding is that InternVL 2.5 benefits from increasing frame counts, whereas InternVL 2.0 often degraded when frame count increased [2412.05271].

Finally, the paper emphasizes **preservation of pure-language ability**. It reports that InternVL 2.5 mitigates the pure-language degradation observed in InternVL 2.0: for example, **InternVL2.5-2B** averages **48.4**, which is **+0.8** over its base LLM, whereas **InternVL2-2B** averages **39.2**, **-2.1** below its base LLM; **InternVL2.5-78B** averages **72.9**, **+1.4** over its base LLM [2412.05271]. This claim is directly tied to stronger text instruction data and stricter filtering.

## 6. Limitations, transfer, and historical significance within the InternVL family

InternVL 2.5’s limitations are stated with unusual clarity for a model-release paper. The system remains weaker than **GPT-4o** on **WildVision**, which the authors interpret as reflecting weaker performance on longer, preference-sensitive outputs [2412.05271]. It also trails leading proprietary systems on **OlympiadBench**, and the paper explicitly notes that **MMVet v2** remains a gap [2412.05271]. Although CoT behavior is improved, the paper says data filtering alone cannot fully remove repetitive behavior, possibly because some noise is inherited from base-LLM pretraining [2412.05271]. Hallucinations also remain, especially in longer outputs [2412.05271].

At the same time, adjacent InternVL-family papers show how InternVL 2.5-era design principles transfer beyond the base benchmarks. The **Mini-InternVL** work demonstrates that a compact derivative using only **1B–4B** parameters can preserve InternVL-family behavior through a strong **InternViT-derived** vision backbone, **dynamic resolution**, a simple **MLP projector**, and staged alignment plus instruction tuning [2410.16261]. Its **Mini-InternVL-4B** variant is reported as retaining about **90%** of the average benchmark performance of **InternVL2-Llama3-76B** while using about **5%** of the parameters, with performance defined over **MMMU (val)**, **MathVista (testmini)**, **AI2D**, **ChartQA**, **DocVQA**, **InfoVQA**, **OCRBench**, **MMBench-EN**, and **MMBench-CN** [2410.16261]. This suggests that InternVL-family capabilities are not solely a product of scale, but also of architectural continuity and training discipline.

The autonomous-driving report based on **InternVL-1.5** is similarly informative because it illustrates how the family’s high-resolution tiling and conversational formatting can be specialized without changing the core model. In that work, six camera images are resized to **896×448**, concatenated into a **2688×896** canvas, paired with a system prompt describing the camera layout, and full-parameter fine-tuned on **DriveLM-nuScenes**, producing a final single-model leaderboard score of **0.6002** [2412.07247]. Later Mini-InternVL results on autonomous driving, medical imaging, and remote sensing extend the same philosophy: domain adaptation is treated primarily as **data reformulation plus supervised fine-tuning**, not architectural branching [2410.16261].

Historically, InternVL 2.5 is also the immediate baseline against which **InternVL3** defines itself. InternVL3 keeps the same **ViT-MLP-LLM** macro-architecture and **pixel unshuffle**, reuses InternVL 2.5 corpora, inherits techniques such as **random JPEG compression**, **square loss re-weighting**, and **multimodal packing**, but replaces the older multi-stage post-hoc adaptation emphasis with **native multimodal pre-training** and adds **V2PE**, **MPO**, and faster infrastructure [2504.10479]. In benchmark terms, InternVL3 generally improves on InternVL 2.5—for example, **MMMU 72.2 vs 70.0** at **78B**—but not uniformly: on **visual grounding**, **InternVL2.5-78B** remains slightly stronger than **InternVL3-78B** (**92.3 vs 91.4**) [2504.10479]. This confirms that InternVL 2.5 should not be seen as a superseded anomaly. Rather, it is the mature culmination of the post-hoc modular InternVL paradigm and the direct predecessor against which the family’s next training philosophy was measured.

In that sense, InternVL 2.5’s lasting significance lies in the specific combination it stabilized: **large InternViT backbones**, **simple projector-based fusion**, **dynamic 448-pixel tiling**, **response-length-aware loss weighting**, **aggressive data filtering**, **broad multimodal instruction mixtures**, and **test-time CoT scaling** [2412.05271]. These features persisted either directly or by explicit contrast in subsequent InternVL-family work, making InternVL 2.5 a pivotal reference point for understanding the modern InternVL line.

Source: https://www.emergentmind.com/topics/internvl-2-5