GLaVE-1.2M: Synthetic Video QA Dataset

• GLaVE-1.2M is a large-scale synthetic training dataset that provides fine-grained video captions and QA pairs derived from 15,814 multi-scene videos.
• It is constructed using a global-local aligned pipeline with vision expert integration and GPT-4o refinement to ensure detailed, context-consistent supervision.
• The dataset serves as a key resource for Video-LLM instruction tuning, demonstrating improved downstream performance on benchmarks compared to prior models.

Searching arXiv for the cited papers and topic variants to ground the article. GLaVE-1.2M is primarily presented as the large-scale synthetic training dataset introduced with "GLaVE-Cap: Global-Local Aligned Video Captioning with Vision Expert Integration" for fine-grained video understanding and Video-LLM instruction tuning. In that usage, it is the training counterpart to GLaVE-Bench and is described in rounded form as containing 16K high-quality fine-grained video captions and 1.2M related question-answer pairs; more precise counts reported elsewhere in the same record are 15,814 videos, 15,814 captions, and either 1,176,410 or 1,146,818 QA items, depending on whether one reads the dataset-construction or training-details section (Xu et al., 14 Sep 2025). The supplied literature also associates the same or a closely aligned name with the 1.2M-utterance Project Euphonia corpus for disordered-speech ASR research, so the term is not uniform across records (Jiang et al., 2024).

1. Placement within the GLaVE-Cap framework

Within the GLaVE-Cap system, GLaVE-1.2M functions as a supervision resource derived from the same global-local captioning and QA-generation pipeline used to build the evaluation benchmark GLaVE-Bench. The paper motivates this pipeline by arguing that the prevailing local-to-global paradigm for video detailed captioning produces less detailed and contextual-inconsistent captions because there is no mechanism to ensure fine-grained captions and because the interaction between local and global captions is weak. GLaVE-Cap addresses those issues with two core modules: TrackFusion, which enables comprehensive local caption generation by leveraging vision experts to acquire cross-frame visual prompts coupled with a dual-stream structure, and CaptionBridge, which establishes local-global interaction by using global context to guide local captioning and adaptively summarizing local captions into a coherent global caption. GLaVE-1.2M is the training-scale consequence of that framework rather than a separate annotation philosophy (Xu et al., 14 Sep 2025).

The dataset is framed as a way to transfer the benefits of GLaVE-Cap into Video-LLM training. The stated goal is to provide richer, more comprehensive supervision than prior open-source video instruction datasets, especially for multi-scene, fine-grained, and object-aware understanding. In that sense, the dataset occupies an intermediate position between captioning methodology and downstream instruction tuning: the captioning pipeline is the generator, and the resulting QA corpus is the training artifact.

2. Source pool, filtering, and reported scale

GLaVE-1.2M is built from the LLaVA-Video-178K source pool, which comprises 178K high-resolution, untrimmed videos from ten different sources, including HD-VILA-100M, ActivityNet, and NeXT-QA. The filtering pipeline keeps videos between 30 and 180 seconds, removes videos with fewer than 2 or more than 10 scenes using PySceneDetect, and then uses GPT-4o to retain only videos with stable camera motion, clear thematic focus, and no frequent scene transitions. After filtering, the retained set contains 15,814 videos totaling 376 hours (Xu et al., 14 Sep 2025).

Aspect	Reported value	Notes
Source pool	LLaVA-Video-178K	178K high-resolution, untrimmed videos
Retained videos	15,814	After duration, scene-count, and GPT-4o filtering
Total duration	376 hours	Post-filter corpus size
Video captions	15,814	One per retained video
QA pairs in construction section	1,176,410	Dataset-construction count
Open-ended QA items used in training	1,146,818	Training-details appendix count

Two numerical conventions coexist in the record. The abstract and main text summarize the resource as “1.2M related question-answer pairs,” while the dataset-construction section reports 1,176,410 QA pairs and the training-details appendix reports 1,146,818 open-ended QA items used for training. The paper does not explicitly resolve that discrepancy. A plausible implication is that “GLaVE-1.2M” is best treated as a rounded dataset label rather than a single exact cardinality.

3. Annotation pipeline and supervision format

The annotation pipeline follows the same general GLaVE-Cap and QA-generation procedure used for GLaVE-Bench. The method first generates fine-grained captions and scene-level hints from a video, then uses those captions to generate question-answer pairs, and finally converts those into multiple-choice question-option pairs for benchmarking. For GLaVE-1.2M, however, the training data retained is not the caption text itself: the appendix states that the dataset used for student-model training consists of 1,146,818 open-ended QA items, and that caption data is excluded from training because it is relatively small in volume and excessively long, which the authors believe could hurt learning (Xu et al., 14 Sep 2025).

The curation logic is more explicit for GLaVE-Bench than for the full training corpus, but it is still relevant because the broader dataset is said to be generated using the Sec. 4.1 method. In that process, captions and scene hints are generated first; question-answer pairs are then refined by GPT-4o through splitting non-atomic questions and rewriting them to reduce answer leakage and improve neutrality; human verification is then applied to ensure correctness, reduce ambiguity, and confirm consistency between questions and scenes. For GLaVE-Bench specifically, this yields 6,491 retained question-option pairs after curation, with approximately 230 human annotation hours spent. The paper does not describe an equally detailed separate human-review pipeline for all of GLaVE-1.2M. This distinction matters when interpreting the dataset’s trustworthiness: the refinement style is shared, but benchmark-level human-review statistics are not directly claimed for the full 1.2M-scale training set.

4. Instruction tuning and reported downstream effects

GLaVE-1.2M is used as instruction-tuning data for GLaVE-7B, a student model built on Qwen2.5-VL-7B. The training details state that all models are trained on 405,752 image samples from LLaVA-One-Vision plus the relevant video data, and that the GLaVE-1.2M video portion used in training consists of 1,146,818 open-ended QA items. Training uses 64 Ascend 910B2 devices across 8 nodes, with GRAD_ACC_STEP set to 16 for a global batch size of 1,024, and runs for 2,000 iterations over about 60 hours. No new loss function is introduced specifically for GLaVE-1.2M beyond standard instruction-tuning (Xu et al., 14 Sep 2025).

The principal empirical evidence for the dataset comes from student-model comparisons. On GLaVE-Bench, GLaVE-7B achieves 79.28 accuracy, compared with 76.31 for the base Qwen2.5-VL-7B, 75.50 for Qwen2.5-VL-7B-Vript, 76.35 for Qwen2.5-VL-7B-ShareGPTVideo, and 75.92 for Qwen2.5-VL-7B-LLaVA-Video. The paper identifies this as the best performance on GLaVE-Bench among the compared student models. On other benchmarks, GLaVE-7B shows a small drop relative to the base model, which the authors attribute to catastrophic forgetting, but they emphasize that the drop is smaller than for other fine-tuned models. Their interpretation is that GLaVE-1.2M provides more detailed and comprehensive supervision than the other open-source training datasets and therefore preserves general capability better while improving fine-grained video understanding.

5. Relation to GLaVE-Bench and to prior video datasets

GLaVE-1.2M and GLaVE-Bench are complementary rather than interchangeable. GLaVE-Bench is the evaluation benchmark, designed to test fine-grained video captioning and understanding using manually refined multi-scene videos, scene hints, and a dense set of QA pairs. It contains 55 manually selected videos and is described as featuring 5X more queries per video than existing benchmarks, covering diverse visual dimensions to facilitate reliable evaluation. GLaVE-1.2M is much larger and is intended for model training. The paper states that the benchmark was created to evaluate the quality of captioning systems and the usefulness of generated captions, whereas GLaVE-1.2M was created to train a student model and to demonstrate that supervision produced by GLaVE-Cap transfers into downstream gains (Xu et al., 14 Sep 2025).

The dataset is explicitly positioned against both older training corpora and more recent synthetic datasets. Compared with older training sets that rely on metadata, ASR transcripts, or weakly aligned captions, GLaVE-1.2M is built on detailed VLM-generated supervision with explicit emphasis on object counts, spatial relations, contextual consistency, and scene-level grounding. Compared with ShareGPT4Video, LLaVA-Video, and Vript, the authors claim that their pipeline more effectively handles multi-scene videos and better preserves fine-grained details. Relative to evaluation sets such as VidCapBench, Video-MME-S, and MVBench, it is not a benchmark but a training corpus meant to improve the ability to answer detailed questions about videos, especially in cases requiring long-range and multi-scene understanding.

A further layer of evidence comes indirectly from ablations of the generation pipeline. Removing the dual-stream structure reduces accuracy by 5.01%; removing adaptive scene splitting reduces accuracy by 21.70%; and removing visual prompts mainly hurts counting-related sub-tasks. The paper also states that the overview caption improves temporal coherence, even if the overall benchmark score does not always change dramatically. This suggests that the supervision quality of GLaVE-1.2M is tied to the specific design of the caption generator from which it was produced.

6. Naming ambiguity and disambiguation from similarly named resources

The name GLaVE-1.2M is not stable across the supplied literature. One record uses it for the video QA training corpus associated with GLaVE-Cap (Xu et al., 14 Sep 2025). Another states that GLaVE-1.2M should be understood as the 1.2M-utterance Project Euphonia corpus, a dataset for automatic speech recognition research on disordered English speech. In that description, the corpus contains over 1.2M utterances from ~2000 speakers, includes 352,130 human-reviewed and corrected transcripts, covers over 75% of speakers with metadata, and spans etiologies including ALS, Parkinson’s disease, Down syndrome, laryngectomy, vocal cord paralysis, hearing impairment, multiple sclerosis, and cerebral palsy (Jiang et al., 2024).

That second usage is methodologically unrelated to the video-captioning dataset. Project Euphonia is framed as a Google initiative to improve ASR of disordered speech, with emphasis on speaker diversity, transcript correction, speech-language-pathologist labels, and the limitations of off-the-shelf VAD on impaired speech. A plausible implication is that searches or secondary references using only the string “GLaVE-1.2M” require domain disambiguation before the dataset can be identified correctly.

A third, distinct name in adjacent literature is GLOVE-1.2M, also written GlOVE-1.2M, which appears in hybrid approximate nearest-neighbor search rather than in captioning or ASR. In HQANN, GLOVE-1.2M is a public ANNS benchmark dataset to which synthetic structured constraints are added, and the paper reports that HQANN reaches 99% Recall@10 in about 50 microseconds on GLOVE-1.2M even with thousands of attribute constraints (Wu et al., 2022). Despite the numerical suffix and near-homographic spelling, this benchmark is separate from both the GLaVE-Cap video dataset and the Project Euphonia speech corpus.

In current usage, the most specific technical meaning of GLaVE-1.2M in the supplied material is therefore the video QA training corpus introduced with GLaVE-Cap: 15,814 filtered long videos, one caption per video, and approximately 1.15M to 1.18M generated QA pairs for fine-grained, multi-scene Video-LLM instruction tuning. The principal caveats are the unresolved QA-count discrepancy, the absence of a separately detailed full-corpus human-review protocol analogous to the benchmark description, and the existence of incompatible name associations in other records.