VideoAVE: Video-to-Text AVE Benchmark

• VideoAVE is a video-to-text attribute extraction benchmark that provides a large-scale e-commerce dataset across 14 product domains with 172 unique attributes.
• It utilizes a CLIP-based Mixture of Experts filtering system to ensure semantic consistency between product videos and their textual metadata.
• Empirical findings demonstrate that while dynamic video data enhances extraction for visually salient attributes, open attribute-value pair extraction remains a challenge.

VideoAVE denotes a video-to-text Attribute Value Extraction benchmark for e-commerce: the first publicly available video-to-text e-commerce AVE dataset across 14 different domains and covering 172 unique attributes (Cheng et al., 15 Aug 2025). It was introduced to address a gap in which existing AVE datasets are primarily limited to text-to-text or image-to-text settings, lacking support for product videos, diverse attribute coverage, and public availability. In a distinct literature, a 2025 paper on Audio-Visual Localization states that it “establishes a new video-centric paradigm for audio-visual event localization (VideoAVE) research,” using the label in a different sense from the e-commerce dataset that formally bears the title “VideoAVE” (Choi et al., 7 Jul 2025).

1. Problem setting and terminological scope

Attribute Value Extraction (AVE) is important for structuring product information in e-commerce. In prior work, most existing AVE research focused on text-to-text or image-to-text settings, whereas product listings frequently include videos that capture temporal and dynamic features, such as the use of a spray bottle. This creates a setting in which systems limited to static text or images can miss important implicit or dynamic attributes (Cheng et al., 15 Aug 2025).

Within this formulation, VideoAVE addresses video-to-text AVE rather than audio-visual event localization, audio-visual speech recognition, or video action detection. Its target output is structured product information in the form of attribute-value pairs derived from product videos. The benchmark therefore belongs to multimodal information extraction and video vision-language modeling, even though its name overlaps with the “video-centric” usage reported in the AVL literature.

A common misconception is that video-to-text AVE is merely image-to-text AVE with additional frames. The benchmark’s design argues against that simplification: it is motivated precisely by dynamic and implicit visual evidence that may not be recoverable from a single image, while also showing that video input does not uniformly dominate other modalities for every attribute.

2. Dataset scope, domains, and curation

VideoAVE is constructed as a large-scale, multi-attribute benchmark. It covers 14 major product domains, 172 unique visually perceivable attributes, and a refined dataset of 224,000 training samples and 25,000 evaluation samples. Each video is annotated with an average of 3.43 attribute-value pairs. The data source is the Amazon Review Dataset, filtered for entries with MP4-format videos and multiple, video-perceivable attributes (Cheng et al., 15 Aug 2025).

Task-oriented pruning is central to the dataset definition. Attributes not visually inferable, subjective or ambiguous attributes answerable by yes/no, and long descriptions were removed; the final attribute set was obtained with a combination of a GPT-4 prompting strategy and human inspection. Samples with only one attribute-value pair were excluded in order to ensure multi-attribute coverage. Post-filtering and cleaning, the dataset is presented as a realistic, large-scale, and diverse testbed for AVE.

Aspect	Specification
Domains	14 major product domains
Attributes	172 unique, visually perceivable attributes
Scale	224,000 training; 25,000 evaluation
Annotation density	Average 3.43 attribute-value pairs
Example domains	Appliances, Arts, Automotive, Baby, Beauty, Clothes, Grocery, Industrial, Music, Patio, Pet, Phones, Sports, Toys

The domain spread matters because AVE in e-commerce is not restricted to a single ontology or product family. A plausible implication is that benchmark performance reflects both visual understanding and schema generalization: systems must recover values across heterogeneous categories such as Color, Material, Shape, Pattern, Power Source, and Mounting Type, rather than optimizing for a narrow attribute inventory.

3. CLIP-MoE filtering and data quality control

A defining technical component of VideoAVE is the post-hoc CLIP-based Mixture of Experts filtering system, CLIP-MoE, introduced to remove mismatched video-product pairs. The stated motivation is that real-world product metadata can be noisy, with titles that do not match the visual content of the associated video. The filtering ensemble uses X-CLIP, ViCLIP, and VideoCLIP, so that each video-title pair receives multiple similarity scores from contrastive vision-LLMs (Cheng et al., 15 Aug 2025).

For each pair $(v_i, t_i)$, each model $m$ computes a similarity score $s_i^{(m)}$. Scores are z-score normalized per model, and only pairs where at least $K$ of the $M$ models produce a normalized similarity score greater than a threshold $\tau$ are retained. In the paper’s notation,

$$\mathcal{D}' = \left\{(v_i, t_i) \mid \sum_{m=1}^{M} \mathbb{I}(\tilde{s}_i^{(m)} > \tau) \geq K\right\}.$$

This filtering stage is not merely a preprocessing detail. It defines the semantic reliability of the benchmark by enforcing consistency between video evidence and textual product identity before downstream attribute extraction is evaluated. This suggests that VideoAVE is positioned not only as a model benchmark but also as a data-engineering contribution, where retrieval-style cross-modal verification is used to stabilize supervision quality.

4. Benchmark tasks, prompting, and evaluated models

VideoAVE defines two benchmark tasks. The first is Attribute-Conditioned Value Prediction, in which a model is given a list of attributes and a product video and must generate the value for each attribute. The second is Open Attribute-Value Pair Extraction, in which only the product video is provided and the model must identify relevant attributes and generate their corresponding values. The benchmark evaluates both closed-schema extraction and open-schema discovery (Cheng et al., 15 Aug 2025).

The attribute-conditioned task uses the prompt template: “Your task is to extract the attributes of the product in this video. Answer it in this format only: ‘attribute1’: ‘value1’, ...”. Evaluation uses fuzzy matching F1, intended to accommodate natural language variation. The open extraction task is described as more stringent because the model must decide what to look for, not just what the value is.

The reported benchmark includes several state-of-the-art video vision LLMs: Video-LLaVA, VideoLLaMA3, InternVideo2.5, and Qwen2.5-VL. In addition, Qwen2.5-VL is fine-tuned on VideoAVE to assess the benefit of task adaptation. This benchmarking choice situates VideoAVE within the current VLM regime rather than within older task-specific extraction architectures.

A useful distinction emerges between the two tasks. Attribute-conditioned value prediction primarily tests value grounding under known slots, whereas open attribute-value pair extraction also tests attribute proposal and salience selection. A plausible implication is that the second task is more sensitive to temporal modeling failures, attention diffusion, and ontology uncertainty.

5. Empirical findings, error modes, and modality comparisons

The benchmark’s main empirical conclusion is that video-to-text AVE remains a challenging problem, particularly in open settings, and that there is still room for developing more advanced VLMs capable of leveraging effective temporal information (Cheng et al., 15 Aug 2025). Across evaluated models, video input is beneficial for many visually salient attributes, including item form, shape, and dynamic usage, but all models struggle to reliably detect both attributes and corresponding values in the open setting.

Fine-tuning improves performance: models fine-tuned on VideoAVE notably outperform zero-shot versions, especially in open attribute-value pair extraction. At the same time, the analysis identifies persistent failure modes. Subjective or ambiguous attributes, such as color shades and ambiguous materials, remain difficult; videos can also introduce noise or distractions through extraneous visual content. These findings matter because they delimit what current VLMs actually recover from video, as opposed to what benchmark designers may hope they recover.

The modality comparison is especially informative. The paper reports that video is better than images and text for most dynamic or highly visual attributes, but also notes exceptions: for attributes better described in the title or difficult to visually determine, text sometimes prevails. This directly counters the assumption that adding video uniformly improves AVE. The benchmark instead portrays modality utility as attribute-dependent and conditioned by perceptual observability.

Another misconception addressed by the results is that open extraction is simply a harder version of the conditioned task by degree. The observed performance gap suggests a structural difference: conditioned prediction constrains the output space, whereas open extraction compounds perception, schema induction, and generation.

6. Relation to video-centric multimodal research

Although VideoAVE is an e-commerce extraction benchmark, its emphasis on temporal information places it within a broader shift toward sequence-aware multimodal modeling. A separate AVL benchmark, AVATAR, argues that prior methods “neglected temporal dynamics,” introduces four scenarios—Single-sound, Mixed-sound, Multi-entity, and Off-screen—and presents TAVLO, a model with factorized spatial-temporal attention that explicitly integrates temporal information (Choi et al., 7 Jul 2025). In adjacent areas, transformer-based video front-ends have replaced 3D convolution in AV-ASR (Serdyuk et al., 2022), JoVALE has introduced actor-centric fusion of audio, visual, and scene-descriptive information for Video Action Detection (Son et al., 2024), and AVE-CLIP has used a multi-window temporal transformer for audio-visual event localization at different temporal scales (Mahmud et al., 2022).

This broader context suggests that VideoAVE’s stress on effective temporal information is not incidental. It aligns with a recurring conclusion across multimodal video research: static or frame-level association is often inadequate when the target variable depends on motion, usage, source transitions, or temporally localized evidence. In VideoAVE, that principle appears in product semantics rather than in sound localization or speech recognition, but the methodological pressure is similar.

In that sense, VideoAVE serves two roles. It is a concrete dataset-and-benchmark contribution for e-commerce AVE, and it is also evidence for a wider research agenda in which video understanding must move beyond frame sampling toward temporally selective, attribute-aware reasoning. The benchmark’s reported limitations—especially in open extraction—indicate that current VLMs still underutilize the temporal structure that motivated the dataset in the first place.