---
title: 'MVQ-60: Multi-Hop VideoQG Dataset'
url: https://www.emergentmind.com/topics/mvq-60-dataset
type: topic
---

# MVQ-60: Multi-Hop VideoQG Dataset

MVQ-60 is a large-scale dataset constructed to benchmark multi-hop video question generation (VideoQG) requiring reasoning across temporally separated video segments. Developed for the evaluation and training of algorithms capable of compositional and multi-hop understanding in vision-language tasks, MVQ-60 provides structured, high-quality question–answer pairs derived from multi-step interactions with long-form TV episode content. The dataset is instantiated via an automated pipeline over TVQA⁺ annotations using formal criteria for segment alignment and semantic overlap, resulting exclusively in two-hop questions that demand integrating facts across distinct portions of the video narrative.

## 1. Formal Specification

MVQ-60 adopts a rigorous multi-hop construction framework. Let the base set \(\{(q_i, a_i, m_i)\}_{i=1}^N\) consist of zero-hop question, answer, and metadata triples from TVQA⁺, with metadata \(m_i = (e_i, s_i)\) denoting episode and segment. Multi-hop (MVQ-2) questions are defined by pairing two zero-hop instances within the same episode but different segments (\(e_1 = e_2, s_1 \neq s_2\)), and ensuring that the answer to the second occurs as a substring in the first question:
\[
\text{Match}(q_1, q_2) = 1 \text{ iff } e_1 = e_2 \wedge s_1 \neq s_2,
\]
\[
\text{Overlap}(q_1, a_2) = 1 \text{ iff } a_2 \text{ is substring of } q_1.
\]
Let \(q_{\mathrm{merged}} = q_1 \setminus a_2 + q_2\), where the span \(a_2\) is replaced by the second question \(q_2\). The principle generalizes to MVQ-\(k\) formulations via
\[
q^{(1)} = q_1, \quad q^{(i)} = q^{(i-1)} \setminus a_i + q_i, \quad i = 2...k.
\]
MVQ-60 contains only two-hop (\(k = 2\)) questions following this merge procedure. The full pseudocode detailing this operation is presented in Appendix Section A.1 (Algorithm 1) of [2511.08348].

## 2. Automated Construction Pipeline

MVQ-60 eschews manual annotation in favor of a scalable algorithmic merger strategy, closely inspired by the MUSIQUE textual multi-hop generation protocol [Trivedi et al. ’22]. Beginning with TVQA⁺’s corpus of 152,545 zero-hop QA pairs, filtering is first applied to retain only those with concise answers (\(\leq 3\) words) and readable questions (\(\leq 15\) words), thus forming \(\mathcal{Q}_{\mathrm{filtered}}\). For each episode, every possible pair of QA triples from distinct segments is examined; if the answer from the second is found in the text of the first question, the formal merge operation is executed to produce a two-hop question. Notably, no balancing for answer-type diversity is applied, leveraging TVQA⁺’s inherent variance spanning character, object, location, and action types. Approximately 60,000 unique questions result from this deterministic multi-hop aggregation.

## 3. Composition, Splits, and Statistical Profile

MVQ-60 covers 21,793 video clips (average duration 75.9 s), sampled from six disparate television series: “Friends,” “The Big Bang Theory,” “How I Met Your Mother,” “House M.D.,” “Grey’s Anatomy,” and “Castle.” Collectively, these form 460 hours of video data and supply the source for both questions and answers.

The dataset comprises:
- 60,000 two-hop (MVQ-2) questions.
- 60,000 corresponding gold-standard answers, inherited from original TVQA⁺ entries.
- Average question length: ~27 words; average answer length: ≤3 words; answer types span character names, object labels, noun phrases, and yes/no categorical forms.

Splits are administered at the episode level for non-overlapping partitions:
| Split     | Percentage | Approximate Question Count |
|-----------|------------|---------------------------|
| Training  | 80%        | ≈48,000                   |
| Validation| 10%        | ≈6,000                    |
| Test      | 10%        | ≈6,000                    |

A plausible implication is that episode-level separation mitigates information leakage between splits, thereby supporting robust generalisation evaluation.

## 4. Annotation Protocol and Quality Assessment

Quality control has been systematically executed via expert annotation and automatic semantic validation. A random sample of 200 merged questions was scored along six axes—Fluency, Relevance, Multi-Hop Reasoning, Engagingness, Factual Correctness, Inclusiveness—by three trained annotators using a 0–3 scale. Achieved mean scores:
- Fluency: 2.92
- Multi-Hop Reasoning: 3.00
- Engagingness: 2.80
- Factual Correctness: 3.00

Cohen’s κ inter-rater agreement of 0.72 indicates substantial consistency. Additional automatic metrics, including BERTScore F1 (0.79) and GPT-5 self-evaluation (Fluency 2.88, Multi-Hop 2.82, Factual 2.74) on another 200-question subset, further substantiate grammaticality, cross-segment reasoning, and factual grounding.

This suggests MVQ-60 substantially meets its multi-hop reasoning objectives and offers reliable input for algorithmic benchmarking under vision-language paradigms.

## 5. Limitations and Domain Constraints

Despite its scale and procedural rigor, MVQ-60 has enumerated limitations:
- **Domain bias:** Exclusivity to six mainstream English-language TV shows; other informal, documentary, educational, or non-scripted video genres are not represented.
- **Reasoning depth:** Only two-hop chains are included; deeper compositional reasoning (≥3 hops) is untested.
- **Input modalities:** Dataset input is restricted to visual frames and transcript; features such as raw audio, object detection, and external knowledge sources are omitted.
- **Language:** All content is English-only; cross-lingual generalisation is infeasible.
- **Error propagation:** Any inaccuracies in zero-hop extraction potentially manifest in merged questions.

A plausible implication is that transferability and cross-modal robustness may be restricted, emphasizing the need for interpretive caution when generalizing results.

## 6. Research Applications and Prospective Extensions

MVQ-60 underpins several research trajectories:
- End-to-end multi-hop VideoQA model training, where the objective is answering as well as generating multi-hop questions over video.
- Benchmarking vision-language transformers on tasks requiring compositional, cross-segment reasoning.
- Pretraining or adaptation for downstream tasks encompassing video narrative understanding, retrieval, and dialogue.

Recommended directions for future dataset evolution include:
- Multilingual question generation to probe cross-lingual reasoning over video.
- Construction of higher-hop (\(k>2\)) multi-hop chains via iterative merge logic.
- Expansion to diverse domains (e.g., lecture, sports, surveillance footage).
- Tri-modal input representations integrating video, transcript, and audio features.
- Explicit balancing by question type to support fair and granular evaluation.

MVQ-60 is publicly accessible at https://github.com/AnupamPandey199949/VideoChain and represents, to date, the most rigorously constructed, large-scale benchmark for multi-hop question generation in the vision-language research field. Its algorithmic pipeline, annotation protocol, and resultant dataset properties collectively facilitate progress in multi-hop video reasoning and generation, while highlighting methodological considerations for future multimodal, multistep QA benchmarks.

Source: https://www.emergentmind.com/topics/mvq-60-dataset