ViExam: Vietnamese Multimodal Exam Benchmark

• ViExam is a benchmark for evaluating vision-language models on real Vietnamese exam questions, combining text with images and diagrams in authentic educational settings.
• It spans seven distinct academic domains, requiring models to integrate OCR, diagram interpretation, and subject-specific knowledge.
• Empirical findings reveal that while state-of-the-art models lag behind human performance, specialized approaches and human collaboration can boost accuracy.

ViExam is a benchmark and empirical study for evaluating vision-LLMs (VLMs) on real Vietnamese multimodal exam questions. It was introduced to test whether VLMs trained predominantly on English data can handle low-resource, curriculum-level, cross-lingual multimodal reasoning in authentic educational settings. The benchmark contains 2,548 multimodal questions drawn from genuine Vietnamese exam materials, and the associated study compares closed-source VLMs, open-source VLMs, a thinking VLM, and human test-takers on a multiple-choice question-answering task over seven academic domains (Dang et al., 19 Aug 2025).

1. Scope, motivation, and position among exam benchmarks

ViExam was created to address a specific gap in multimodal evaluation: most existing benchmarks and VLM assessments are English-centric, emphasize general vision-language tasks such as captioning or generic visual question answering, and provide limited coverage of low-resource languages, particularly for multimodal educational content. ViExam therefore focuses on Vietnamese educational assessments that are both genuinely multimodal and drawn from real examination practice rather than synthetic or toy data (Dang et al., 19 Aug 2025).

The benchmark is explicitly educational and domain-diverse. It spans Mathematics, Physics, Chemistry, Biology, Geography, Driving Test, and IQ Test, and it is designed so that models see the original Vietnamese question together with its visual components and must choose the correct option. This exam-taking setup distinguishes ViExam from evaluations centered on open-ended captioning or broad visual understanding, because the task couples OCR, diagram interpretation, domain knowledge, and discrete decision-making under realistic exam conventions (Dang et al., 19 Aug 2025).

Within the broader literature on exam-style multimodal benchmarks, ViExam is narrower in language coverage but more targeted in linguistic and human-comparison objectives than multilingual resources such as EXAMS-V, which contains 20,932 multiple-choice questions across 20 school disciplines in 11 languages from 7 language families (Das et al., 2024). ViExam’s contribution is thus not breadth across many languages, but depth on Vietnamese multimodal educational reasoning together with direct human-vs-model comparisons, cross-lingual prompting analysis, and human-in-the-loop experiments (Dang et al., 19 Aug 2025).

2. Corpus design, sources, and multimodal representation

ViExam is a curated benchmark of 2,548 Vietnamese exam questions collected from real Vietnamese exams, including official school exams, national and standardized exams where relevant, official Vietnamese driving license exam questions, and standardized IQ-type questions used in Vietnamese context. The questions are educational and real-world, and they combine text with images, diagrams, tables, maps, chemical structures, road signs, and other visual elements (Dang et al., 19 Aug 2025).

Dataset construction follows a filtering and normalization pipeline. Only questions with clearly defined correct answers are retained. Questions must be standalone and self-contained, and priority is given to multimodal items with images, diagrams, or tables, although text-only questions can be included when they still require reasoning or language understanding. Ambiguous wording and disputed official answers are removed, and the resulting set is organized to reflect a mix of difficulty levels and question types such as concept recall, problem solving, multi-step reasoning, visual perception, and spatial reasoning (Dang et al., 19 Aug 2025).

The representation preserves the original Vietnamese exam format as closely as possible. Vietnamese text is fully preserved, including diacritics, with standard UTF-8 encoding. Images are extracted, cleaned, and stored at sufficient resolution. LaTeX formulas are retained where present or transcribed when needed. Each item is annotated with its domain or subject, its answer key, and additional metadata where applicable, such as grade level, exam type or year, and difficulty labels if available. The benchmark has no explicit train/validation/test split and is used purely for evaluation (Dang et al., 19 Aug 2025).

The seven subject areas induce heterogeneous multimodal demands. Mathematics includes geometric diagrams, coordinate grids, and function plots; Physics includes circuits, force diagrams, and motion schematics; Chemistry includes molecular structures and laboratory apparatus; Biology includes anatomical diagrams, cell structures, and ecosystem charts; Geography includes maps and topographic diagrams; Driving Test questions use road signs and intersection scenes; and IQ Test items use pattern sequences and visual analogies. Many math and physics questions include LaTeX-style equations such as $v = \frac{\Delta s}{\Delta t}$, $F = ma$, and $W = F \cdot s$, while chemistry items may include equations such as $\mathrm{2H_2 + O_2 \rightarrow 2H_2O}$ (Dang et al., 19 Aug 2025).

3. Task formulation and evaluation protocol

The benchmark is formulated as a multiple-choice question-answering task. For each question, the model receives one or more images together with the original Vietnamese question text and answer options. The required output is a single selected answer option, typically one of $\{A, B, C, D\}$, although other formats are normalized to option selection. Models may be allowed to produce free-form reasoning in some settings, particularly for thinking models, but the final answer must map to the discrete exam option (Dang et al., 19 Aug 2025).

Evaluation uses accuracy as the primary metric, computed both overall across all 2,548 questions and per domain across the seven subject categories. There is no partial credit: each question is either correct or incorrect. Model outputs are post-processed to extract an explicit option label, with case and formatting variations normalized. If a response contains multiple options or no identifiable valid option, it is marked incorrect or resolved by taking the first clearly indicated option according to the evaluation pipeline (Dang et al., 19 Aug 2025).

Prompting is standardized across model families. Closed-source systems with role separation typically receive a system prompt instructing them to answer Vietnamese multiple-choice exam questions, analyze the image if present, and select the best option. User prompts present the Vietnamese question and options directly. Models are generally instructed to respond only with the chosen letter, although some experiments permit reasoning before the final choice. This protocol is intended to mirror actual exam-taking rather than open-ended explanation or generation (Dang et al., 19 Aug 2025).

A notable protocol variant is cross-lingual prompting. In that setting, the question text and answer options remain in Vietnamese exactly as in the source material, while only the instructional part of the prompt is changed to English. The purpose is to test whether models trained heavily on English instructions benefit from English task framing when solving Vietnamese content. ViExam reports that this strategy does not improve performance (Dang et al., 19 Aug 2025).

4. Models evaluated and quantitative findings

ViExam evaluates a range of vision-LLMs, including closed-source or proprietary state-of-the-art systems, open-source VLMs, and the thinking VLM o3. The closed-source group includes representative systems such as GPT-4V and other strong commercial multimodal models accessed through APIs with default or lightly customized prompts. The open-source group includes LLaVA variants, Qwen-VL variants, InternVL, and similar multimodal models, typically evaluated without fine-tuning on ViExam in zero-shot or few-shot prompt settings. These systems generally have visual encoders based on CLIP, ViT, or related architectures and are trained predominantly on English and Chinese image-text pairs with limited Vietnamese coverage (Dang et al., 19 Aug 2025).

The central quantitative result is that ViExam is difficult for present-day VLMs. Across all 2,548 questions, closed-source state-of-the-art VLMs achieve 57.74% mean accuracy, open-source models achieve 27.70% mean accuracy, human average performance is 66.54%, human best performance is 99.60%, and the thinking VLM o3 reaches 74.07% accuracy. Most VLMs therefore underperform average human test-takers, while o3 is the only reported model exceeding the human average and still remains far below the best human solver (Dang et al., 19 Aug 2025).

The study reports domain-level regularities rather than a single uniform difficulty profile. Mathematics and Physics require multi-step quantitative reasoning and interpretation of formulas and graphs; open-source models perform poorly here, while commercial models and o3 do better but still make notable errors. Chemistry and Biology depend strongly on domain-specific knowledge and visual diagrams, and VLMs struggle with chemical equations and complex biological structures. Geography requires map reading, climate-chart interpretation, and spatial reasoning. Driving Test items combine sign recognition with right-of-way and scene interpretation, and performance varies depending on whether a model has useful road-sign priors. IQ Test items probe pattern recognition and visual analogies, where models sometimes succeed at local pattern matching but still fail when multiple rules must be composed (Dang et al., 19 Aug 2025).

The thinking VLM o3 improves across nearly all domains and is reported to excel in Physics, Math, and IQ, where careful reasoning appears beneficial. Even so, it still struggles in some Chemistry and Biology questions that combine specialized knowledge with precise Vietnamese language understanding. The paper notes only limited use of formal statistical tests, but it also states that the differences in mean accuracies are large enough to be clearly meaningful (Dang et al., 19 Aug 2025).

5. Human baselines, cross-lingual prompting, and human-in-the-loop collaboration

Human evaluation is integral to ViExam’s design. The study includes Vietnamese-speaking participants with relevant educational backgrounds, such as high school graduates or university students, tested under exam-like conditions with time constraints and no external resources. Accuracy is computed in the same way as for models. The reported human average is 66.54%, and the top human solver reaches 99.60%, which is close to perfect performance on the benchmark (Dang et al., 19 Aug 2025).

This human baseline sharpens the interpretation of model scores. ViExam does not merely show that Vietnamese multimodal exam questions are hard; it shows that the difficulty is not symmetric between humans and models. Current commercial VLMs remain below average human exam-takers, open-source models lag substantially further behind, and even the strongest thinking VLM falls well short of the best human performance. This suggests that the unresolved bottleneck is not only general reasoning depth, but also the interaction among Vietnamese language understanding, visual grounding, and exam-specific conventions (Dang et al., 19 Aug 2025).

Cross-lingual prompting is examined directly by changing only the instructional language from Vietnamese to English while leaving the question content in Vietnamese. On average, this decreases accuracy by about 1 percentage point for state-of-the-art VLMs. The reported interpretation is that the language mismatch between English instructions and Vietnamese content may introduce subtle confusion, and that simply switching instructions to English does not unlock stronger Vietnamese multimodal performance. ViExam treats this as evidence of limits in cross-lingual generalization for high-stakes, structured tasks (Dang et al., 19 Aug 2025).

The benchmark also studies human-in-the-loop collaboration. In this setting, a human may rephrase the question, eliminate obviously wrong options, or provide brief hints about relevant visual elements, while the VLM still makes the final choice. This collaboration improves VLM performance by about 5 percentage points. The qualitative account is that humans compensate for VLM weaknesses in Vietnamese linguistic nuance, distractor elimination, and visual focus, while VLMs remain useful for some forms of rapid pattern recognition and straightforward formula application (Dang et al., 19 Aug 2025).

6. Failure modes, interpretive significance, and future directions

ViExam includes a systematic error analysis. Common failure modes include language misinterpretation, such as errors on Vietnamese negation, comparator words, or technical terms; diagram misreading, including incorrect identification of geometric relations or missed cues in circuit diagrams; math and physics reasoning errors in multi-step calculations or graph interpretation; chemistry and biology knowledge gaps, such as incorrect stoichiometry or confusion over diagrammed structures; geography map-based mistakes involving regions, direction, or topographic symbols; driving-test failures in sign recognition or right-of-way reasoning; and IQ-test failures when multiple transformation rules operate simultaneously (Dang et al., 19 Aug 2025).

Representative examples illustrate the interaction of these failure modes. A math question asking for the maximum of a function from a plotted graph may be answered incorrectly because the model misreads the peak. A physics problem involving $v = \frac{\Delta s}{\Delta t}$ and a distance-time table may fail because the wrong interval is selected or the ratio is miscomputed. A chemistry question requiring a balanced equation such as $\mathrm{2H_2 + O_2 \rightarrow 2H_2O}$ may reveal weak stoichiometric reasoning. A driving question involving an intersection and a stop sign may be answered incorrectly when the model ignores the sign or misapplies Vietnamese right-of-way rules. The study’s qualitative conclusion is that VLMs often get either the visual perception or the reasoning approximately right, but not both simultaneously, and they are especially fragile when precise Vietnamese wording mediates the reasoning step (Dang et al., 19 Aug 2025).

The paper interprets these results as evidence that cross-lingual transfer of multimodal reasoning is incomplete. Current VLMs trained largely on English and other major languages do not automatically achieve human-level performance on Vietnamese educational content. The recommended directions for improvement are more Vietnamese multimodal training data, including textbooks, annotated diagrams, exam solutions, and road-sign datasets; better cross-lingual alignment at both text and image levels; domain-specific fine-tuning for subjects such as physics, math, and driving tests; and architectural improvements for formal symbolic computation, diagram understanding, and layout-aware visual processing (Dang et al., 19 Aug 2025).

ViExam is also presented as a template for future benchmark development. Proposed extensions include adding more domains such as literature, history, and economics; incorporating other question types, including open-ended short answers and proof-style or explanation questions; and extending the approach to other languages with similar educational exam formats. Code and data are available through the project website, enabling use for benchmarking new VLMs, testing domain-specific fine-tuning, studying human–AI collaboration, and investigating cross-lingual multimodal reasoning in low-resource educational settings (Dang et al., 19 Aug 2025).