VARCO-VISION-2.0: Bilingual Open-Weight VLM

• VARCO-VISION-2.0 is an open-weight bilingual vision-language model supporting both Korean and English, with explicit multi-image input and layout-aware OCR.
• It employs a LLaVA-OneVision architecture with Qwen3 backbones and SigLIP2 encoder, optimizing token efficiency for varied image resolutions.
• Available in 14B and 1.7B variants, the model excels in Korean-localized tasks, deployability, and structured multimodal output for research applications.

VARCO-VISION-2.0 is an open-weight bilingual vision-LLM (VLM) for Korean and English developed by NCSOFT and presented as a successor to VARCO-VISION-14B. The model family is defined by three closely linked aims: stronger Korean-localized multimodal performance, explicit support for multi-image understanding, and layout-aware OCR that predicts both textual content and spatial location. The released family contains a full-scale 14B model and a lightweight 1.7B model, both implemented in the Hugging Face Transformers format and compatible with vLLM. In the accompanying technical report, the 14B model is described as achieving 8th place on the OpenCompass VLM leaderboard among models of comparable scale, while an external medical benchmark study identifies VARCO-VISION-2.0-14B as the strongest Korean-specialized model in its evaluation but still substantially behind the strongest proprietary and general-purpose open models on Korean medical multimodal reasoning (Cha et al., 12 Sep 2025, Choi et al., 14 Feb 2026).

1. Lineage, scope, and released variants

VARCO-VISION-2.0 belongs to the VARCO-VISION family introduced as an open-source Korean-English bilingual VLM line oriented toward image-text understanding, image-grounded generation, OCR, grounding, and referring. The earlier VARCO-VISION paper centered on a single released model, VARCO-VISION-14B, built from Qwen-2.5-14B-Instruct, SigLIP, and an MLP projector in a LLaVA-OneVision-style configuration, and paired that release with five Korean multimodal benchmarks. VARCO-VISION-2.0 preserves the family’s bilingual and open-weight orientation while shifting to Qwen3 backbones, SigLIP2, multi-image support, and a revised preference-optimization stage (Ju et al., 2024, Cha et al., 12 Sep 2025).

Two variants are released. VARCO-VISION-2.0-14B is the full-scale model intended for server/cloud deployment, research, and high-quality production inference. VARCO-VISION-2.0-1.7B is optimized for on-device deployment such as smartphones and PCs, with lower deployment cost and smaller memory footprint. The technical report states that both models are available on Hugging Face as NCSOFT/VARCO-VISION-2.0-14B and NCSOFT/VARCO-VISION-2.0-1.7B (Cha et al., 12 Sep 2025).

The designation bilingual is central rather than incidental. The model is explicitly trained and evaluated for both Korean and English, with Korean query regeneration from English tasks, bilingual OCR data, Korean-localized contextual preference data, and benchmark coverage in both languages. The designation open-weight is equally consequential: the model parameters are publicly released rather than restricted to API access, enabling local execution, adaptation, benchmarking, and deployment in data-sensitive environments (Cha et al., 12 Sep 2025).

2. Architecture and multimodal interface

VARCO-VISION-2.0 is built on the LLaVA-OneVision architecture and consists of three main components: a LLM backbone, a vision encoder, and a multimodal connector/projector. The LLM is Qwen3-14B for the 14B variant and Qwen3-1.7B for the lightweight variant. The vision encoder is SigLIP2 with patch-16 configuration, replacing the earlier family’s SigLIP patch-14 setting. A two-layer MLP connector projects visual features into the language-model embedding space (Cha et al., 12 Sep 2025).

For a 384 × 384 image, the report gives a concrete token-count tradeoff: patch-16 produces $24^2 = 576$ visual tokens, whereas patch-14 produces $27^2 = 729$. The change is framed as an efficiency choice, based on the observation that prior work suggested token count has limited effect on overall VLM quality. The model also uses the AnyRes strategy from LLaVA-OneVision, with tiled or cropped views and resolution-aware aggregation for arbitrary image resolutions. Training-time maximum grid settings are $1 \times 1$ in Stage 1, $4 \times 4$ in Stage 2, and $6 \times 6$ in Stages 3 and 4; with encoder resolution 384, the $6 \times 6$ grid corresponds to an effective maximum resolution of $384 \times 6 = 2304$ (Cha et al., 12 Sep 2025).

One of the major architectural changes relative to VARCO-VISION-14B is explicit multi-image input support. The technical report states that the model supports multiple images simultaneously and that an internal document QA dataset contains up to 12 Korean-English images per sample. For multi-image scenarios in Stage 3, a fixed-size image representation strategy is used to control sequence length and remain compatible with the context window. The vocabulary size is reported as approximately 150K tokens, which is operationally significant because large-vocabulary logit computation becomes a memory bottleneck during training (Cha et al., 12 Sep 2025).

A distinctive feature of the family is its structured text interface for OCR and grounding. The model uses <gro> for grounding and <ocr> for OCR. Grounding outputs take the serialized form

$\texttt{<obj>\{object</obj><bbox>\{x1\}, \{y1\}, \{x2\}, \{y2\}</bbox>\}$

with normalized coordinates in $[0,1]$. OCR outputs take the form

$\texttt{<char>\{word</char><bbox>\{x1\}, \{y1\}, \{x2\}, \{y2\}</bbox>\}$

again with normalized coordinates. The report states that OCR words are segmented at whitespace boundaries and ordered by y-coordinate clustering, with reading order top to bottom, left to right. The grounding and OCR mechanisms are therefore implemented as sequence-generation problems with text-serialized spatial outputs rather than through a separate detector or coordinate-regression head (Cha et al., 12 Sep 2025).

3. Four-stage curriculum and training system

VARCO-VISION-2.0 is trained with a four-stage curriculum, inspired by Qwen2.5-VL and InternVL3, and the report states total training consumption of approximately 6.5B text tokens, 30.4B image tokens, and 36.9B total tokens. The stage-wise allocation is highly skewed toward Stage 3, which uses 5.7B text tokens and 25B image tokens, for 30.7B total (Cha et al., 12 Sep 2025).

Stage	Text tokens	Image tokens
1	4M	560M
2	760M	4.7B
3	5.7B	25B
4	61M	93M

Stage 1, Feature Alignment Pre-training, trains only the MLP connector while freezing both the vision encoder and the LLM. The supervision consists of filtered image-caption data with real-world images and concise English descriptions of key objects. This stage is used to align the independently pretrained visual and linguistic modules and stabilize image-to-text mapping (Cha et al., 12 Sep 2025).

Stage 2, Basic Supervised Fine-tuning, jointly trains the full model in single-image settings with AnyRes at relatively low resolutions. The described data categories include re-captioned image, chart, and table datasets, together with a bilingual OCR dataset containing Korean and English text, diverse fonts and styles, and both collected and synthetically generated images. The paper provides a standardized OCR-style prompt instructing section-by-section reading from top to bottom and left to right. This stage is intended to build broad multimodal instruction-following, chart and table understanding, and bilingual OCR reading (Cha et al., 12 Sep 2025).

Stage 3, Advanced Supervised Fine-tuning, is the principal capability-building phase. It introduces higher-resolution perception, multi-image reasoning, document QA, grounding, OCR with localization, and strengthened bilingual instruction following. Korean queries are regenerated from English prompts and answers, and human annotators enrich prompts with output-format instructions such as “Answer the question using a single word or phrase” and “Explain the solution process step by step and provide the final answer.” The report describes an in-house document QA dataset with up to 12 Korean-English images per sample, plus fine-grained grounding and word-level OCR-localization data (Cha et al., 12 Sep 2025).

Stage 4, Preference Optimization, applies Direct Preference Optimization (DPO) to improve safer responses, reduced hallucination, better cultural and contextual localization, and improved user-intent alignment. A major change from VARCO-VISION-14B is that the previous model updated only LLM layers in Stage 4, whereas VARCO-VISION-2.0 unfreezes the entire model. The paper describes three preference-data categories: general anti-hallucination image preferences, safety preferences and refusals, and Korean-localized culturally aware responses. It does not print the DPO formula in the report itself, but it explicitly identifies DPO as the method (Cha et al., 12 Sep 2025).

Language preservation is treated as a first-order objective. The report attributes strong text-only retention to the Qwen3 base model, the use of compact, high-quality multimodal datasets, and an efficient training strategy that does not excessively erode language competence. This emphasis extends a design principle already visible in the original VARCO-VISION paper, which incorporated text-only data during later stages to preserve backbone language proficiency (Ju et al., 2024, Cha et al., 12 Sep 2025).

The training system is unusually explicit about memory management. The technical report states that training used a single node with 8×H100 GPUs for about 700 wall-clock hours, or roughly 5,600 GPU-hours. Memory-efficiency techniques include FSDP, activation checkpointing, 8-bit Adam, Liger kernel, CPU offloading in DPO, and custom logit chunking. For the 14B model, the authors also use merge-train-merge, merging several Stage 3 checkpoints to initialize Stage 4 and then merging Stage 4 checkpoints to obtain the final model. The 1.7B model does not use checkpoint merging; instead, its vision encoder is initialized from the 14B model after Stage 3, and ablations show that shared initialization helps while freezing hurts (Cha et al., 12 Sep 2025).

4. Evaluation profile beyond medicine

The 14B model is reported to rank 8th on the OpenCompass VLM leaderboard among models with fewer than 20B parameters, and its English benchmark average is given as 77.2, compared with 68.1 for VARCO-VISION-1.0-14B, 78.4 for InternVL3-14B, 77.9 for Ovis2-16B, and 76.0 for Qwen2.5-VL-7B. Selected 14B English benchmark results include RealWorldQA 74.6, SEEDBench_IMG 78.0, Q-Bench1_VAL 79.9, A-Bench_VAL 79.5, ChartQA_TEST 84.2, MMMU_VAL 61.9, MathVista 73.2, DocVQA_TEST 90.9, TextVQA_VAL 80.4, and OCRBench 86.9. The report explicitly characterizes the model as strongest in real-world perception and spatial tasks, but not best in knowledge-heavy or document-centric tasks (Cha et al., 12 Sep 2025).

On Korean multimodal benchmarks, the 14B model has an average of 80.7, ahead of InternVL3-14B 80.2, Qwen2.5-VL-7B 77.2, Ovis2-16B 71.4, and VV-1.0-14B 77.3. The key 14B scores are K-MMBench_DEV 87.7, K-MMStar 63.6, K-SEED 77.2, K-LLaVA-W 96.5, and K-DTCBench 78.3. The K-LLaVA-W result is described as best by a wide margin and is used to support the claim of improved Korean generation and dialogue ability. The relatively low K-DTCBench score is presented as evidence that structured document, table, and chart understanding remains a comparative weakness (Cha et al., 12 Sep 2025).

On text-only benchmarks, the 14B model has an average of 75.5, compared with 74.4 for Ovis2-16B, 71.9 for InternVL3-14B, and 60.2 for VV-1.0-14B. Selected scores are MMLU 77.9, MT-Bench 89.8, KMMLU 57.5, KoMT-Bench 78.3, and LogicKor 74.0. This continues a pattern already emphasized by the earlier VARCO-VISION work, where the authors argued that multimodal training should not destroy text competence and reported unusually strong text-only retention for a VLM (Ju et al., 2024, Cha et al., 12 Sep 2025).

OCR is one of the most favorable parts of the 2.0 evaluation. On recognition accuracy benchmarks, the report gives VV-2.0-14B as CORD 97.1, ICDAR2013 95.7, ICDAR2015 79.4, and Average 90.7. The 1.7B model is also strong, with Average 88.6, and even slightly exceeds the 14B model on ICDAR2013 with 95.9. The report states that the 14B model nearly matches CLOVA OCR on average, beats it on CORD, and clearly surpasses PaddleOCR and EasyOCR on average. The OCR evaluation further reports that upscaling to a minimum longer side of 2304 px significantly improves OCR because it matches the finest grid tokenization supported by the model (Cha et al., 12 Sep 2025).

The 1.7B variant occupies a distinct position. Its English average is 68.3, below InternVL3-2B 70.5 and Ovis2-2B 71.7, but it is described as especially strong in physical-world visual understanding, with RealWorldQA 66.8, LLaVABench 77.3, Q-Bench1_VAL 72.3, MathVista 61.1, and OCRBench 83.0. On Korean multimodal benchmarks, the 1.7B model has an average of 65.4, which the report identifies as best among lightweight models, though it suffers notably on K-MMStar 40.8 due to response-format violations under Korean instructions (Cha et al., 12 Sep 2025).

5. KorMedMCQA-V and Korean medical multimodal reasoning

An important external evaluation of VARCO-VISION-2.0-14B appears in KorMedMCQA-V, a Korean medical licensing-exam-style multimodal multiple-choice question answering benchmark for VLMs. The benchmark contains 1,534 multimodal questions drawn from the 2012–2023 Korean Medical Licensing Examination, paired with 2,043 images, and each item is a 5-way MCQA problem in which the model receives the full question stem, five answer options, and one or more associated images, then outputs a single option label. The evaluation is zero-shot, closed-book, uses a single prompt template, requires output in JSON format, and scores predictions by exact match on the option label; for open-source models, the paper runs three random seeds (42, 43, 44) and reports mean ± std (Choi et al., 14 Feb 2026).

Within this benchmark, the medical paper treats VARCO-VISION-2.0 as a Korean-specialized and open-source model. It does not provide a bespoke architectural description of VARCO-VISION-2.0; instead, it identifies the evaluated checkpoints as NCSOFT/VARCO-VISION-2.0-1.7B and NCSOFT/VARCO-VISION-2.0-14B, both run with vLLM, under the same unified protocol as the other open-source baselines. This makes the benchmark comparison operationally consistent, while the authors also note broader comparability caveats arising from each model’s own default processor and default or recommended generation settings (Choi et al., 14 Feb 2026).

The headline result is that VARCO-VISION-2.0-14B achieves 43.2% accuracy, reported in Appendix Table A3 as $27^2 = 729$0. This makes it the best Korean-specialized model in the study, ahead of A.X-4.0-VL-Light 41.8, Kanana-1.5-V-3B-Instruct 30.7, HyperCLOVAX-SEED-Vision-Instruct-3B 26.3, and VARCO-VISION-2.0-1.7B 24.4. At the same time, it is 36th of 51 overall, with 35 models scoring higher and only 15 scoring lower, placing it in the lower third of the leaderboard. The strongest proprietary model, Gemini-3.0-Pro, reaches 96.9%; the strongest open-source model, Qwen3-VL-32B-Thinking, reaches 83.7%; and the strongest medical-specialized model, Lingshu-32B, reaches 66.1% (Choi et al., 14 Feb 2026).

The modality breakdown shows generally low and uneven performance: XRAY 42.4, CT 35.4, ECG 46.3, US 39.6, Endoscopy 41.3, NST 32.6, PBS 48.1, MRI 44.4, and Other 48.5. Its weakest modalities are NST and CT, and the paper notes that the model is below the overall 51-model average in every listed modality. By image count, the scores are 1 image 43.7, 2 images 43.0, and 3+ images 33.3, indicating a 10.4-point drop from single-image to 3+ image questions. This is consistent with the paper’s broader conclusion that all models degrade on multi-image questions and that integrating evidence across multiple medical images remains an open challenge (Choi et al., 14 Feb 2026).

A particularly informative analysis comes from the combined KorMedMCQA-Mixed benchmark. For Mixed-2022, VARCO-VISION-2.0-14B scores Text 59.3, Vision 46.6, Total 52.7. For Mixed-2023, it scores Text 57.3, Vision 45.9, Total 51.5. The medical benchmark paper explicitly interprets this as evidence that Korean-specialized models show large and consistent text-over-vision gaps, and states that VARCO-VISION-2.0-14B exhibits +12.7 and +11.4 percentage point text advantages, identifying medical visual reasoning as the primary weakness of Korean-specialized VLMs. Under KMLE-derived pass criteria, which require $27^2 = 729$1 per exam session and $27^2 = 729$2 overall, the model fails both years (Choi et al., 14 Feb 2026).

The medical benchmark therefore provides a narrow but important correction to the more favorable general-domain evaluation profile. VARCO-VISION-2.0-14B is competitive in several bilingual and OCR-heavy settings, but the evidence in KorMedMCQA-V suggests that Korean language specialization and general multimodal competence do not by themselves yield strong KMLE-style multimodal medical reasoning (Choi et al., 14 Feb 2026).

6. Limitations, reproducibility, and broader significance

The technical report is explicit about several limitations. Instruction robustness is weak, with outputs sensitive to superficial formatting changes such as whitespace and newlines. The model remains weaker on knowledge-intensive tasks and document-centric tasks than on perception-heavy tasks. Referring capability is said to be weaker than in the previous VARCO-VISION model. In OCR, the report states that while content understanding can extrapolate with larger grids, precise text localization fails sharply beyond training grid settings; with $27^2 = 729$3 grids and longer token sequences, OCRBench improves but localization accuracy may drop to zero. The report also notes continued Korean cultural data scarcity and benchmark artifacts caused by response-format violations rather than pure understanding failure (Cha et al., 12 Sep 2025).

Reproducibility is only partial. The report provides the architecture family, key backbone choices, token counts by stage, context lengths, hardware, training time, and major optimization methods, but it does not publish a complete list of training datasets or data proportions. It also does not state the license in the report text provided here. The earlier VARCO-VISION paper had a similar profile: checkpoint and benchmark release were clear, but the full training-data recipe, hyperparameters, and some supervision formats were not exhaustively documented (Ju et al., 2024, Cha et al., 12 Sep 2025).

In the broader landscape of open VLMs, VARCO-VISION-2.0 occupies a specific niche. Its strongest documented contributions are an open-weight Korean-English multimodal model, explicit multi-image capability, strong spatial grounding and layout-aware OCR, and a paired 1.7B deployment-oriented variant. The empirical record supports the claim that it is substantially stronger than VARCO-VISION-14B across many general and Korean-localized benchmarks, especially on Korean generation and OCR. At the same time, the KorMedMCQA-V results indicate that language specialization alone is insufficient for high-end medical multimodal reasoning and that performance in demanding professional domains remains much lower than that of stronger proprietary and reasoning-oriented open models. This suggests that VARCO-VISION-2.0 is best understood not as a universal frontier model, but as a technically distinctive bilingual open VLM family whose main value lies in Korean-English multimodal coverage, deployability, and structured visual-text output interfaces rather than across-the-board dominance (Cha et al., 12 Sep 2025, Choi et al., 14 Feb 2026).