VaseVQA: Visual QA for Greek Pottery
- The paper introduces VaseVQA, a novel visual question answering benchmark that integrates 2D imagery and 3D models to analyze ancient Greek pottery attributes.
- It employs supervised fine-tuning combined with diagnosis-guided reinforcement learning to enhance factual recall and expert reasoning on attributes like fabric, technique, and provenance.
- The benchmark supports digital heritage preservation by enabling detailed morphological and iconographic analysis, with its 3D extension providing robust full-object reasoning using GLB models and rotation videos.
Searching arXiv for the VaseVQA papers to ground the article in the latest preprints. VaseVQA is a domain-specific visual question answering benchmark and multimodal agent framework for ancient Greek pottery, designed to probe both factual recall and expert reasoning over attributes such as fabric, technique, shape, provenance, date, attribution, and decoration. In its original form, it centers on 2D image-based open-ended VQA and diagnosis-guided reinforcement learning; in its 3D extension, VaseVQA-3D, it becomes a benchmark for vision–language reasoning over GLB-format vase models, canonical multi-view renders, and 360° rotation videos, reflecting the archaeological importance of full-object morphology and iconographic placement (Ge et al., 21 Sep 2025, Zhang et al., 6 Oct 2025).
1. Scope, domain, and research problem
VaseVQA was introduced to address a persistent weakness of general-purpose multimodal LLMs in cultural-heritage analysis: brittle reasoning and shallow pattern-matching on tasks such as style classification, historical attribution, and compositional description. The benchmark is explicitly organized around ancient Greek pottery, where answers often require integrating visual cues such as shape, decorative program, and technique with culturally grounded knowledge such as chronology, workshop practices, and provenance (Ge et al., 21 Sep 2025).
The original benchmark contains 31,773 images collected through collaborations with archaeological institutions, museums, and cultural heritage centers. Within that resource, a curated single-view subset contains 11,693 images paired with 93,544 visual question–answer pairs, with eight questions per image. The single-view subset is split into 9,354 training images and 2,339 test images; a distinct validation split is not mandated, and practitioners typically hold out part of the training data or use internal validation during supervised fine-tuning. Images are drawn from museum and field documentation, including complete objects, fragments, and burial contexts, with repositories such as the Beazley Archive Pottery Database explicitly mentioned among the sources (Ge et al., 21 Sep 2025).
VaseVQA-3D extends the same domain to object-centric 3D reasoning. It is presented as the first 3D VQA benchmark for ancient Greek pottery and contains 664 GLB-format 3D vase models paired with structured question–answer data and enhanced captions. The 3D dataset contains 4,460 question–answer pairs, averaging 6.72 per vase, with a 70\%/15\%/15\% train/val/test split of 420/90/90 models. The extension is motivated by the fact that many diagnostically important attributes cannot be resolved from a single 2D photograph, including exact vessel form, handle attachments, foot and lip profiles, and decoration on the reverse side (Zhang et al., 6 Oct 2025).
2. Dataset design, taxonomy, and annotation structure
The original VaseVQA benchmark is VQA-centric and open-ended. Its core taxonomy spans seven factual and expert types—Fabric, Technique, Shape, Provenance, Date, Attribution, and Decoration—plus an eighth “Genera/overall details” slot used in some splits. The benchmark includes concise factual prompts such as “What is the fabric of the vase?”, “What is the technique of the vase?”, and “What is the shape name of the vase?”, as well as compositional templates that require joint reasoning across multiple attributes. Examples given in the source include iconography + shape + period for attribution and provenance + technique + inscription for historical context (Ge et al., 21 Sep 2025).
Annotations are organized in a lightweight JSON format with conversational pairs: 8 Labels are derived from academic papers, expert annotations by archaeologists and historians, and structured descriptions of materials, motifs, techniques, chronology, and context. Annotators label material or fabric, manufacturing technique, shape classes, provenance or origin, date ranges in BCE, attribution to painters or workshops, and restoration markers (Ge et al., 21 Sep 2025).
VaseVQA-3D retains the archaeological orientation but standardizes the structured VQA component as “What is the [attribute] of the vase?” across six core dimensions: Fabric, Technique, Shape, Dating, Decoration, and Attribution. The dataset statistics also track Provenance and a Caption category, for eight reported types in total. After filtering, each of the six core attributes accounts for approximately 14.9\% of questions, while Attribution and Provenance account for 6.3\% and 4.4\%, respectively. Each 3D vase is rendered in four canonical views—front, back, top, bottom—and paired with GPT-4o–enhanced descriptive captions that merge metadata and domain language into concise archaeological descriptions (Zhang et al., 6 Oct 2025).
The 3D extension is explicit about modality. Its core representation is GLB meshes, with geometry encoded in the mesh and appearance carried via mesh textures. It does not provide separate point clouds or voxel grids. Canonical multi-view images and 360° rotation videos are used in model training and evaluation; the paper does not specify camera intrinsics or extrinsics and does not claim rotation invariance, but instead normalizes viewpoint through standardized renders and rotation videos (Zhang et al., 6 Oct 2025).
3. Learning methodology in the original VaseVQA system
The accompanying multimodal agent for the original benchmark is VaseVL, an SFT-then-RL system built on a general-purpose multimodal backbone such as Qwen2.5-VL 3B. The central methodological claim is that VaseVL “turns evaluation into supervision.” The procedure first performs supervised fine-tuning to obtain a reference policy , then probes post-SFT performance by question type to identify brittle categories, and finally applies GRPO-based reinforcement learning with type-conditioned, compositionality-oriented rewards targeted at those weaknesses (Ge et al., 21 Sep 2025).
The supervised fine-tuning objective is given as
The reinforcement-learning stage optimizes a reward-regularized objective anchored to the SFT policy:
Reward construction combines lexical and semantic agreement,
with type-specific mixing coefficients,
and diagnosis-guided amplification for underperforming types,
This design is intended to preserve factual recall while improving reasoning on categories such as Attribution and Decoration (Ge et al., 21 Sep 2025).
Evaluation in the original benchmark is type-aware rather than uniform across all question classes. Factual categories—Fabric, Technique, Shape, Provenance, and Attribution—use ANLS-based accuracy. Date is evaluated by a specialized parser for temporal ranges and standardized BCE formats. Decoration uses BLEU@1. The official scripts also report an “Overall” score aggregating per-type results under the benchmark protocol (Ge et al., 21 Sep 2025).
4. Empirical behavior of VaseVQA and VaseVL
The benchmark was designed to expose a domain gap between general-purpose multimodal models and culturally specialized reasoning. Zero-shot baselines are reported as weak on expert-level questions. On the test split, Qwen2-VL 7B records Fabric 1.69, Technique 24.00, Shape 37.66, Provenance 21.24, Attribution 2.85, Decoration 2.00, and Overall 0.93. LLaVA 7B is reported at Shape 44.60, Attribution 28.41, Decoration 6.28, and Overall 4.78. These results are used to argue that general models lack domain expertise on ancient Greek pottery (Ge et al., 21 Sep 2025).
Instruction-tuned Qwen2.5-VL-SFT (3B) already reaches high factual recall but remains weak on the more compositional categories. The reported SFT scores are Fabric 99.96, Technique 94.99, Shape 83.98, Provenance 71.67, Date 37.96, Attribution 56.96, Decoration BLEU@1 2.57, and Overall 74.25. VaseVL, which adds the reinforcement-learning phase, improves these to Fabric 99.95, Technique 95.93, Shape 83.99, Provenance 73.67, Date 39.87, Attribution 60.83, Decoration BLEU@1 9.82, and Overall 75.71. The source highlights gains of +3.87 points on Attribution and +7.25 BLEU@1 on Decoration, while near-ceiling Fabric performance remains essentially unchanged (Ge et al., 21 Sep 2025).
These results are interpreted in the paper as evidence for diagnosis-guided reward shaping rather than indiscriminate RL. The largest improvements occur on the categories explicitly identified as brittle after SFT, and the GRPO+KL design is described as stabilizing optimization and preventing regression on factual types. The paper further states that VaseVL achieves state-of-the-art results on style-related classification and historical attribution, and that the resource is intended to be reusable for future research on cultural-heritage understanding (Ge et al., 21 Sep 2025).
5. VaseVQA-3D and the shift to full-object reasoning
VaseVQA-3D is motivated by a methodological distinction between object-centric heritage artifacts and generic image benchmarks. For ancient Greek pottery, the exact profile of a rim, foot, or handle attachment can determine shape classification; decorative programs may extend around the full body; and technique or fabric cues may appear on edges, undersides, or interiors. The 3D benchmark operationalizes these needs through GLB meshes, four canonical renders, and 360° rotation videos used during training (Zhang et al., 6 Oct 2025).
The 3D construction pipeline begins from the large-scale 2D VaseVQA image collection and curated 3D references for a validation set called VaseEval. Because many source images are fragments, sketches, or low-quality photographs, the pipeline applies three stages of filtering before 2D-to-3D synthesis. A ResNet-50 binary classifier trained with cross-entropy, Adam at , and 20 epochs removes blurry, dark, and low-resolution inputs, reducing 30,000 images to 13,599, for a retention rate of 45.3\%. A CLIP-based fragment filter using ViT-B/32 and a threshold of 0.1 reduces this set to 6,330 and reports a quality score of 0.156. A further CLIP-based view-selection stage reduces the set to 3,880 images and raises the quality score to 0.234 (Zhang et al., 6 Oct 2025).
From the 3,880 filtered images, TripoSG synthesizes 664 GLB models, corresponding to a 17.1\% success rate. The paper benchmarks TripoSG against Hunyuan3D on VaseEval, a set of 24 ground-truth GLBs from Sketchfab, and chooses TripoSG for large-scale generation because it yields better Chamfer Distance and stronger CLIP-Text alignment: Chamfer Distance 0.1490 versus 0.1515 and CLIP-Text 0.9594 versus 0.9237, although Hunyuan3D has slightly better Normal Consistency. Blender 3.6 is then used to render 360° rotation videos of 16 frames at and 2 fps from the GLB files (Zhang et al., 6 Oct 2025).
The accompanying 3D model, VaseVLM, uses Qwen2.5-VL in 3B and 7B variants as its backbone and does not introduce a new cross-modal fusion module beyond the Qwen2.5-VL framework. Adaptation instead proceeds through a two-stage process. First, LoRA-based supervised fine-tuning is applied to rotation videos and archaeological captions with LoRA rank 8, alpha 32, learning rate , batch size 1 with 16-step gradient accumulation, 2 epochs, and frozen vision encoder parameters. Second, GRPO reinforcement learning is combined with RLVR, or Reinforcement Learning with Verifiable Rewards, in which captions are decomposed into six semantic dimensions—Fabric, Technique, Shape, Dating, Decoration, and Attribution—with weights , 0, 1, 2, 3, and 4 (Zhang et al., 6 Oct 2025).
For dimension 5, the RLVR reward is
6
with cosine similarity and threshold 7. The penalty term is
8
with 9, 0, and 1, and the final reward is
2
bounded to 3. GRPO uses a policy update every 100 samples, learning rate 4, batch size 8, and 10 epochs (Zhang et al., 6 Oct 2025).
6. Results, significance, and future directions
On the VaseVQA-3D test set, the best reported model is VaseVLM-7B-RL, with FID 0.328, CLIP 0.792, 5 21.24\%, 6 11.12\%, 7 3.52\%, and lexical similarity 0.276. Representative baselines include DiffuRank with FID 0.421, CLIP 0.798, 8 16.67\%, 9 2.08\%, and lexical similarity 0.274; Gemini-2.5-flash with FID 0.325, CLIP 0.736, 0 28.57\%, 1 2.20\%, and lexical similarity 0.210; Qwen2.5-VL-3B with FID 0.358, CLIP 0.782, 2 9.38\%, 3 1.04\%, and lexical similarity 0.259; and VaseVL with FID 0.493, CLIP 0.790, 4 10.4\%, 5 2.08\%, and lexical similarity 0.255. The paper reports gains of +12.8\% on 6 and +6.6\% on lexical similarity relative to the previous state of the art on VaseVQA-3D (Zhang et al., 6 Oct 2025).
Ablation results in the 3D paper indicate that RL improves over SFT for both model sizes. For the 7B system, SFT moves from FID 0.332 and lexical similarity 0.272 to RL at FID 0.328 and lexical similarity 0.276. For the 3B system, lexical similarity improves from 0.223 to 0.245. Human expert evaluation by 10 archaeologists rates VaseVLM-7B-RL highest, with an average of 4.57/5, approximately +0.4 over the SFT version and approximately 12.3\% above the best baseline DiffuRank at 4.07 (Zhang et al., 6 Oct 2025).
The practical significance claimed across the VaseVQA line is tied to digital heritage preservation. In the original benchmark, automated VQA can support style classification, historical attribution, and compositional description. In the 3D extension, consistent GLB models, full-surface renders, and captions are described as useful for online exhibits, educational resources, curation, cataloging, and conservation support. Unlike scene-centric 3D VQA datasets, VaseVQA-3D targets object-centric heritage artifacts with precise morphological and iconographic attributes, and it is framed as complementary to recent 3D captioning and QA efforts such as Cap3D, DiffuRank, and LLaVA-3D (Ge et al., 21 Sep 2025, Zhang et al., 6 Oct 2025).
The released resources include code, data, and evaluation scripts at https://github.com/AIGeeksGroup/VaseVQA and https://github.com/AIGeeksGroup/VaseVQA-3D, together with a project site for the 3D benchmark. Reported implementation details include ResNet-50 and CLIP thresholds, TripoSG settings with 7 input, Blender rotation-video parameters, LoRA-SFT hyperparameters, and the GRPO schedule (Ge et al., 21 Sep 2025, Zhang et al., 6 Oct 2025).
The main limitations also differ by stage. For the original benchmark, domain coverage emphasizes funerary and classical vases and includes fragments, and some answers such as provenance may be unavailable. For the 3D extension, only 17.1\% of filtered images could be converted to high-quality GLBs, the pipeline is tailored to Greek vases, and 3D generation dominates a roughly 14-day pipeline, with approximately 13.5 days on a single A100. Planned directions include improving 3D reconstruction success and quality, generalizing the pipeline to other cultural artifacts, developing more efficient training, expanding annotations, and considering links to external knowledge bases for richer attribution and provenance context (Ge et al., 21 Sep 2025, Zhang et al., 6 Oct 2025).
A plausible implication is that VaseVQA could evolve beyond answer-only evaluation toward explicit evidence-grounded reasoning. VistaQA presents a methodological template for a “VaseVQA-style benchmark” in which each answer is coupled to pixel-level segmentation evidence and evaluated by a joint, non-compensatory metric such as GROVE; that extension is not part of the current VaseVQA releases, but it directly matches the need for transparent support of archaeological claims (Azadani et al., 20 May 2026).