CulturalVQA: Benchmarking Cultural Visual QA

• CulturalVQA is a research domain that benchmarks vision-language models’ cultural understanding through diverse datasets and multimodal evaluation protocols.
• It encompasses curated datasets across cuisines, rituals, art, and daily practices, employing multiple QA formats to probe model reasoning.
• Empirical studies highlight performance gaps in under-represented regions and emphasize the need for robust, culturally aware evaluation metrics.

CulturalVQA is a research domain, dataset class, and evaluation paradigm focused on benchmarking and advancing vision-LLMs’ ability to comprehend, reason about, and explain cultural content from diverse global contexts. The term encompasses datasets, metrics, model architectures, and evaluation protocols specifically designed to probe knowledge of culturally distinctive artifacts, practices, symbolism, and narratives in images—ranging from everyday practices and culinary items to heritage material, religious rituals, and art. The core challenge underlying CulturalVQA lies in transcending Western-centric biases, providing equitable coverage and rigorous measurement for high- and low-resource settings, and driving both factual and inferential multimodal cultural understanding.

1. Dataset Construction and Coverage

The culturalVQA paradigm is instantiated through large, systematically curated datasets representing geo- and ethnically diverse cultures. Prominent examples include:

• RICE-VL: 28,000+ VQA samples (True/False, Fill-in-the-Blank, Open-ended) drawn from 7,000 images across 11 ASEAN countries, annotated in 14 cultural domains and paired with a visual grounding dataset of 1,000 image-bounding box pairs covering 95 subcategories (Pranav et al., 1 Dec 2025).
• CulturalVQA (original): 2,378 image-question pairs, with 1–5 human answers each, spanning 11 countries and 5 continents; questions cover food, drinks, clothing, rituals, and traditions, selected using World Values Survey cultural blocs (Nayak et al., 15 Jul 2024).
• Regional/Single-Culture Benchmarks: K-Viscuit (Korean culture, 657 MCQ items, 10 high-level concepts) (Park et al., 24 Jun 2024), TCC-Bench (Traditional Chinese, 860 bilingual MCQs, 8 domains) (Xu et al., 16 May 2025), VietMEAgent’s Vietnamese corpus (91,149 QAs, 12 categories) (Nguyen et al., 12 Nov 2025), IndicVisionBench’s Indian subcontinent suite (37K QAs, 13 topical domains, 11 languages) (Faraz et al., 6 Nov 2025).
• Art and Heritage: VQArt-Bench (14,463 MCQs, 7 reasoning dimensions), focusing on symbolic, relational, and semantic art interpretation (Alfarano et al., 14 Oct 2025).
• Multilingual/Massive Scale: WorldCuisines (1.15M VQA triplets in 30 languages, 9 families, 2,414 dishes, 96 “global foods”) (Winata et al., 16 Oct 2024), CulturalGround (22M VQA pairs, 39 languages, 42 countries) (Nyandwi et al., 10 Aug 2025).
• Multimodal/Visual Grounding: Datasets such as RICE-VL, Seeing Culture (1,093 unique Qs, 3,178 MCQs, segmentations for evidence), and BLEnD-Vis (21,782 MCQs, 16 regions, aligned across text-only and VQA formats) probe spatial reasoning, cross-modal alignment, and robustness to paraphrase or distractor cues (Pranav et al., 1 Dec 2025, Satar et al., 20 Sep 2025, Tan et al., 13 Oct 2025).

CulturalVQA datasets typically combine diverse imaging sources (web, museums, community photos), enlist annotators with deep cultural knowledge, and enforce balance across facets (e.g., clothing, food, festivals, heritage) to avoid skew towards objects or traditions better represented in Western data.

2. Task Formulation and Evaluation Protocols

CulturalVQA tasks fall into several distinctive types, each probing different axes of cultural knowledge:

• Closed-form MCQ: Standard in K-Viscuit, VQArt-Bench, WorldCuisines, etc.; models select one of four or five semantically similar options, with distractors carefully constructed to elicit nuanced reasoning (Park et al., 24 Jun 2024, Winata et al., 16 Oct 2024, Alfarano et al., 14 Oct 2025).
• Open-ended Generative: Models must produce free-form answers, judged via LLM-based metrics or human raters (e.g., CulturalVQA’s use of LAVE, BLEU-4, METEOR, ROUGE-L) (Nayak et al., 15 Jul 2024, Nguyen et al., 12 Nov 2025).
• True/False and FIB: RICE-VL separates factual verification, fill-in, and explanation within the same corpus, enabling fine-grained assessment of text-visual grounding and cultural specificity (Pranav et al., 1 Dec 2025).
• Visual Grounding and Segmentation: Beyond answering, models localize or segment the relevant artifact (IoU, mean-IoU as key metrics)—either via bounding boxes (RICE-VL) or segmentation masks (Seeing Culture) (Pranav et al., 1 Dec 2025, Satar et al., 20 Sep 2025).
• Programmatic Reasoning/Explanation: VietMEAgent demonstrates programmatic decompositions of the QA process (object detection, symbolic KB queries, visual attention), pushing beyond pure answer correctness toward transparent multimodal explanations (Nguyen et al., 12 Nov 2025).

Evaluation relies heavily on metrics that fuse semantic/language match with cultural alignment. For example, RICE-VL’s SEA-LAVE metric aggregates Text Understanding (TU), Cultural Understanding (CU), and Country Identification (CI), giving: $\mathrm{SEA\mbox{-}LAVE} = \frac{TU + CU + \bigl(\tfrac{CI}{2}\bigr)}{3}$ (Pranav et al., 1 Dec 2025). Similarly, BLEnD-Vis and VQArt-Bench employ accuracy and consistency assessments across rephrased, region-swapped, and visually grounded variants (Tan et al., 13 Oct 2025, Alfarano et al., 14 Oct 2025).

3. Empirical Findings and Failure Modes

CulturalVQA benchmarks consistently reveal several empirical regularities in model performance:

• Closed-source models outperform open-source by 15–20 percentage points on average in both overall accuracy and cultural alignment, regardless of region or language (Pranav et al., 1 Dec 2025, Nayak et al., 15 Jul 2024, Park et al., 24 Jun 2024).
• Low-resource regions/countries exhibit the largest deficits. In RICE-VL, Timor-Leste and Laos have SEA-LAVE scores <0.45 versus Thailand’s 0.82 (Claude-3-Opus) (Pranav et al., 1 Dec 2025); in CulturalVQA, Ethiopia/Nigeria lag the US by ≥27 points (Nayak et al., 15 Jul 2024); WorldCuisines shows stronger MCQ drops and near-random OEQ results for non-Latin scripts and under-represented cuisines (Winata et al., 16 Oct 2024).
• Domain and question-type sensitivity: Tangible facets (food, clothing, festivals) outperform abstract ones (rituals, art, key figures), with visual complexity and cultural symbolism being frequent failure points (Pranav et al., 1 Dec 2025, Xu et al., 16 May 2025, Satar et al., 20 Sep 2025, Alfarano et al., 14 Oct 2025).
• Prompt and context sensitivity: Frame prompts (“This is a Southeast Asian setting”) can improve SEA-LAVE by up to 0.28 (Ola model on Thailand from 0.59 to 0.87; RICE-VL), explicit context increases MCQ accuracy (WorldCuisines); adversarial context and paraphrase cause 10–15% drops in accuracy (Pranav et al., 1 Dec 2025, Winata et al., 16 Oct 2024, Tan et al., 13 Oct 2025).
• Visual grounding lags reasoning: Even SOTA VLMs achieve mean-IoU ≈ 0.52 (Qwen2.5-VL in RICE-VL) or 47% (Qwen2.5-VL-7B in Seeing Culture); in many cases, models can choose the correct image but fail to provide spatially faithful evidence masks (Pranav et al., 1 Dec 2025, Satar et al., 20 Sep 2025).
• Model size is not sufficient: Some larger open-source models (e.g., Intern-VL 25B) do not close the gap with smaller closed-source models, especially on under-represented cultures (Nayak et al., 15 Jul 2024, Winata et al., 16 Oct 2024).

4. Benchmarking Innovations and Methodological Advances

CulturalVQA research has advanced benchmarking protocols through:

• Hybrid Human–LLM Question Generation: E.g., K-Viscuit’s semi-automated, cultural expert–VLM workflow for MCQ construction led to higher diversity and less cognitive burden than purely human annotation (Park et al., 24 Jun 2024). VQArt-Bench’s multi-agent LLM pipeline with sequential topic selection, question crafting, distractor refinement, and judging dramatically improves coverage and semantic richness (Alfarano et al., 14 Oct 2025).
• Multimodal Retrieval-Augmented Methods: RAVENEA demonstrated that lightweight VLMs augmented with culture-aware document retrieval outperformed baseline models by >3.2% absolute on cultural VQA; improvements were especially pronounced on under-represented countries (e.g., Nigeria, Indonesia) (Li et al., 20 May 2025).
• Programmatic and Explanation-Driven Pipelines: Program synthesis for both answer and explanation (VietMEAgent) provided substantial gains in cultural accuracy (+0.525 over baseline) and improved transparency in high-complexity domains (e.g., Handicrafts, Daily Life) (Nguyen et al., 12 Nov 2025).
• Fine-Grained Robustness Probes: BLEnD-Vis evaluates not just static accuracy but also robustness to template rephrasing, cross-modal consistency (text–visual), and region-level biases, exposing systematic brittleness in current VLMs (Tan et al., 13 Oct 2025). CultureMix introduces “culture mixing” as a formal challenge—combining multiple conflicting cultural cues within a scene, showing up to 14% country identification accuracy drop under mixed backgrounds (Kim et al., 27 Nov 2025).

Dataset	Countries / Regions	QA Format(s)	Key Metric(s)	Notable Findings
RICE-VL	11 (ASEAN)	TF, FIB, OE, Grounding	SEA-LAVE, IoU	Abstract domains hardest
CulturalVQA	11 (5 continents)	Open-ended, 1–5 references	LAVE, string-match acc.	Africa < West by ~27 pp
K-Viscuit	1 (Korea)	MCQ only	Accuracy	Proprietary > open-source
VQArt-Bench	Art/global	MCQ, 7 taxonomy dimensions	Accuracy (by dimension)	Counting weakest
TCC-Bench	1 (China, bilingual)	MCQ (ZH/EN), explanations	Accuracy (ZH>EN), ablation	Text-only baseline low
WorldCuisines	189 countries	MCQ, OEQ, multilingual	Accuracy, BERTScore	Adversarial context issues
Seeing Culture	7 (SE Asia)	MCQ+segment	VQA acc., mIoU	Acc–IoU gap, subtlety issue
VietMEAgent	1 (Vietnamese)	OE+explanation, programmatic	BLEU-4, Cultural Acc.	Knowledge base critical

5. Implications, Biases, and Open Problems

CulturalVQA exposes critical limitations in contemporary VLMs and points to several enduring challenges:

• Cultural representation bias: Low-resource countries and non-Western domains are under-represented in image–text pretraining corpora, biasing VLM performance. RICE-VL and WorldCuisines both document ~15–20 point accuracy gaps and systematic “default to high-resource” behaviors when models are uncertain (Pranav et al., 1 Dec 2025, Winata et al., 16 Oct 2024).
• Superficial pattern matching: State-of-the-art models often rely on coarse cues (e.g., background, stereotyped attire) rather than fine-grained visual analysis or narrative inference. Purpose-built distractors in K-Viscuit and VQArt-Bench trip up models exploiting shortcuts (Park et al., 24 Jun 2024, Alfarano et al., 14 Oct 2025).
• Transfer and Multilingual Generalization: Even models with massive training scale exhibit language- and region-specific weaknesses; e.g., TCC-Bench sees 77% accuracy for Chinese prompts but as low as 41% for English ones, demonstrating the importance of idiomatic context (Xu et al., 16 May 2025). IndicVisionBench shows ~50% drops for low-resource scripts despite high overall performance (Faraz et al., 6 Nov 2025).
• Model adaptation and robustness: Prompting ("regional framing”), retrieval augmentation, and explicit program-based explanations improve scores, but model robustness to adversarial context, paraphrase, or “culture-mixed” scenes is still weak (Kim et al., 27 Nov 2025, Li et al., 20 May 2025).
• Evaluation depth: Current metrics sometimes conflate string match with true cultural competence; explanation-quality, multimodal evidence, and cross-modal agreement (as in BLEnD-Vis) provide more granular views of model capabilities but are not yet standardized.

6. Prospects and Future Directions

Emerging CulturalVQA research converges on several priority recommendations:

• Expand high-quality, regionally diverse datasets via community partnerships, crowdsourcing, and targeted cultural documentation—broadening representation across both majority and minority cultures (Pranav et al., 1 Dec 2025, Nayak et al., 15 Jul 2024).
• Integrate structured cultural knowledge (ontologies, domain-specific KBs)—as in VietMEAgent or retrieval-augmented approaches (RAVENEA)—for reasoning beyond surface cues (Nguyen et al., 12 Nov 2025, Li et al., 20 May 2025).
• Embrace multilingual, dialectal, and code-mixed settings to model culture-identity coupling and capture regional expressiveness, as shown vital by WorldCuisines and IndicVisionBench (Winata et al., 16 Oct 2024, Faraz et al., 6 Nov 2025).
• Advance explainable VQA via joint answer/explanation objectives and programmatic pipelines to support educational, transparent cultural AI (Nguyen et al., 12 Nov 2025).
• Benchmark for robustness in culture-mixing, adversarial, and compositional scenarios, and prioritize metric development for generative and explanatory competence (Kim et al., 27 Nov 2025, Alfarano et al., 14 Oct 2025).
• Collaborate with humanistic and cultural scholars in dataset creation and annotation, ensuring fidelity, contextual accuracy, and global relevance (Pranav et al., 1 Dec 2025, Xu et al., 16 May 2025).

CulturalVQA thus stands at the intersection of multimodal AI, global cultural studies, and responsible dataset engineering—serving as both a diagnostic stress test and a roadmap for equitable, culturally aware vision-language systems.