--- title: 'VaseVL: Multimodal Ancient Greek Pottery Analysis' url: https://www.emergentmind.com/topics/vasevl type: topic --- # VaseVL: Multimodal Ancient Greek Pottery Analysis VaseVL is a domain-adaptive multimodal system for ancient Greek pottery understanding that appears in two closely related forms in recent literature. In the 2D setting, it is presented as an SFT-then-RL multimodal agent built on a pretrained multimodal LLM and evaluated with the VaseVQA benchmark of 31,773 images and 93,544 question–answer pairs [2509.17191]. In the 3D setting, the same system name is used for an end-to-end pipeline centered on the VaseVQA-3D dataset and the VaseVLM model, extending the task to 664 GLB-format 3D vase models with 4,460 question–answer pairs and domain-adaptive caption generation from multi-view renderings [2510.04479]. Across both formulations, the motivating problem is the same: general VLMs and MLLMs exhibit a substantial domain gap on specialized cultural-heritage tasks, especially for style classification, historical attribution, and archaeologically grounded description. ## 1. Problem Setting and Conceptual Scope The VaseVL line of work is motivated by the claim that analyzing cultural-heritage artifacts remains challenging for MLLMs because general models lack domain expertise, and SFT often overfits superficial patterns, yielding brittle reasoning for authentication and historical attribution [2509.17191]. The 3D extension sharpens this diagnosis by emphasizing severe data scarcity issues and insufficient domain knowledge limitations in 3D vase artifact analysis, particularly because targeted training data had been absent prior to VaseVQA-3D [2510.04479]. Operationally, VaseVL treats ancient Greek pottery understanding as a structured multimodal reasoning problem over recurring archaeological attributes. In the 2D benchmark, these attributes are organized into the question types Fabric, Technique, Shape, Provenance, Attribution, Date, and Decoration. In the 3D benchmark, the construction process begins from six-attribute archaeological metadata—Fabric, Technique, Shape, Dating, Decoration, Attribution—and converts that metadata into fixed-schema question–answer pairs, while also generating rich captions for caption-based retrieval evaluation. This suggests that VaseVL is not merely a generic VQA system specialized by prompt wording; it is explicitly organized around an archaeological ontology embedded in both supervision and reward design. A central conceptual feature is that evaluation is reused as supervision. The 2D system explicitly “turns evaluation into supervision” by probing the SFT model for type-specific weaknesses and then using taxonomy-conditioned rewards during RL to target those deficiencies [2509.17191]. The 3D system adopts a related logic through domain-adaptive SFT followed by reinforcement learning with verifiable rewards (RLVR), where reward is decomposed along six semantic dimensions corresponding to archaeological attributes [2510.04479]. ## 2. Benchmarks and Data Resources The VaseVL ecosystem comprises a large 2D benchmark and a smaller but structurally novel 3D benchmark. | Resource | Scale | Notes | |---|---:|---| | VaseVQA | 31,773 images; 93,544 QA pairs | 8 questions per image; seven question types | | VaseVQA-3D | 664 GLB-format 3D models; 4,460 QA pairs | first 3D visual question answering dataset for ancient Greek pottery analysis | | VaseEval | 24 GLB models | museum-grade subset from Sketchfab for reconstruction validation | VaseVQA contains 31,773 images, of which 11,693 are single-view “standalone” shots, and 93,544 QA pairs. Its splits are Train: 9,534 images → 76,272 QA and Test: 2,339 images → 18,712 QA [2509.17191]. Each vase image is paired with eight canonical questions organized into seven types: Fabric, Technique, Shape, Provenance, Attribution, Date, and Decoration. These categories are intended to probe distinct reasoning skills, ranging from factual classification to descriptive scene understanding. VaseVQA-3D contains 664 high-fidelity GLB-format 3D models of ancient Greek vases spanning shapes such as amphorae, kraters, and lekythoi; decorative programs including geometric, black-figure, red-figure, and white ground; dating from the 8th–4th century BCE; fabrics; and painter attributions [2510.04479]. A small VaseEval subset of 24 museum-grade GLB models from Sketchfab is used to validate 3D reconstruction quality. The 3D dataset is constructed by starting from VaseVQA’s 30 K+ 2D vase images plus six-attribute archaeological metadata, applying three-stage visual filtering, performing 2D→3D conversion with TripoSG, adopting the original structured VaseVQA questions, and enhancing metadata-derived captions with GPT-4o. The resulting benchmarks serve different experimental purposes. VaseVQA is a large VQA-centric benchmark designed to probe deep understanding across a broad taxonomy of archaeological questions. VaseVQA-3D is a targeted 3D benchmark whose narrower scale is offset by explicit multi-view geometric representation and caption-based evaluation. A plausible implication is that the two datasets instantiate complementary notions of domain expertise: one emphasizes taxonomic breadth in 2D, while the other emphasizes modality extension into 3D artifact understanding. ## 3. 2D VaseVL: Architecture, Taxonomy, and Reward Design In the 2D formulation, VaseVL is built in two major phases—Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL)—on top of a pretrained multimodal LLM such as Qwen-2.5-VL [2509.17191]. Lightweight LoRA adapters are attached into the cross-attention layers that fuse visual features from a frozen image encoder such as CLIP-Vision or ViT into the LLM’s language stream. The data flow is: image $x \rightarrow$ frozen image encoder $\rightarrow$ visual tokens $V$; question $q$ and $V \rightarrow$ multimodal transformer layers with LoRA adapters $\rightarrow$ contextualized embeddings; then the LLM decoder autoregressively generates the answer $\hat y$. During SFT, VaseVL is fine-tuned on tuples $(x,q,a^*)$ via maximum-likelihood on the concatenated prompt `"[Image: x] Question: q Answer:"` to obtain a reference policy $\pi_{\rm ref}$. The objective is $$ L_{\rm SFT}(\theta) = - E_{(x,q,a^*)\sim D} \sum_{t=1}^{|a^*|} \log \pi_\theta(a^*_t \mid x,q,a^*_{1$ when $q\in C_{\rm short}$: $$ \tilde R(q;\hat y,a^*) = w(q)\,R_{\rm type}(q;\hat y,a^*). $$ In GRPO, the method samples $K$ answers, computes a per-prompt baseline $\bar r$, forms advantages $A^{(k)} = r^{(k)}-\bar r$, and optimizes $$ \max_\theta E\big[A^{(k)} \log \pi_\theta(\hat y^{(k)}|x,q)\big] - \lambda\,KL(\pi_\theta||\pi_{\rm ref}). $$ Within the VaseVL literature, this mechanism is presented as diagnosis-guided, taxonomy-conditioned reward engineering. Its role is not to replace the benchmark with a single scalar objective, but to preserve per-type evaluation distinctions and feed them back into training. ## 4. VaseVQA-3D Construction Pipeline The 3D extension begins from VaseVQA’s 30 K+ 2D vase images plus six-attribute archaeological metadata: Fabric, Technique, Shape, Dating, Decoration, and Attribution [2510.04479]. A three-stage visual filtering pipeline is then applied. First, a ResNet-50 binary QC classifier removes blurred, dark, or sketchy images. Second, a CLIP ViT-B/32 semantic test rejects fragments by comparing similarity to “complete intact vase” versus “vase fragment.” Third, CLIP view selection picks the single best view per object. After filtering, 2D→3D conversion is performed with TripoSG, selected over Hunyuan3D via VaseEval metrics. Question–answer construction adopts the original VaseVQA structured question pattern—“What is the [attribute] of the vase?”—and a fixed-answer schema derived from verified metadata, yielding 4,460 QA pairs. Caption enhancement then feeds raw metadata plus model IDs into GPT-4o to generate fluent, archaeologically rich descriptive captions. This pipeline is significant because it couples deterministic metadata supervision with synthetic 3D reconstruction and multi-view rendering. The design avoids free-form human captioning at the first stage of dataset construction, instead grounding QA and caption generation in verified archaeological metadata. A plausible implication is that the 3D benchmark prioritizes attribute fidelity and retrieval consistency over unconstrained descriptive diversity. The benchmark also includes illustrative vase-specific QA examples. Shape identification is represented by “What is the shape of the vase?” → “Amphora.” Decoration classification includes “What is the decoration of the vase?” → “A red-figure scene depicting Herakles wrestling the Nemean lion.” Additional examples cover manufacturing technique (“White-ground lekythos.”), dating (“Mid-5th century BCE.”), fabric composition (“High-quality Attic clay.”), and attribution (“The Berlin Painter.”) [2510.04479]. These examples clarify that VaseVQA-3D is designed around archaeologically meaningful answer spaces rather than generic visual question answering prompts. ## 5. VaseVLM in 3D: Base Model and Domain-Adaptive Training In the 3D formulation, the model component is named VaseVLM. Its visual encoder is a frozen ViT from Qwen2.5-VL that consumes 360° rotation video frames, specifically 16 frames at $512\times512$ rendered via Blender [2510.04479]. Cross-modal fusion and the language decoder are provided by the Qwen2.5 LLM backbone, and LoRA adapters of rank 8 with $\alpha=32$ are inserted into cross-attention layers for parameter-efficient fine-tuning. The SFT stage minimizes the standard cross-entropy loss over the target caption tokens: $$ \mathcal{L}_{\text{SFT}} = -\sum_{t=1}^T \log p_\theta(y_t\mid x, y_{