TimeLens-Bench: Refined VTG Benchmark

• TimeLens-Bench is a high-fidelity benchmark suite designed to evaluate temporal localization and event grounding in video with rigorous re-annotation protocols.
• It implements a two-stage diagnose-then-refine pipeline with strict criteria for query clarity, uniqueness, and temporal precision to correct legacy dataset errors.
• The refined benchmarks, spanning multiple datasets, improve model evaluation accuracy and set a new standard for reproducibility in temporal video understanding research.

TimeLens-Bench refers to a set of rigorously designed, high-fidelity benchmarks for evaluating temporal localization and event grounding in video, with a specific focus on resolving severe annotation pathologies that have historically skewed model performance and misled the research community. The suite comprises re-annotated and quality-controlled versions of three canonical Video Temporal Grounding (VTG) datasets—Charades-STA, ActivityNet Captions, and QVHighlights—each enhanced by systematic error detection and correction protocols. Its design directly addresses fundamental deficiencies in previous VTG benchmarks and establishes a new standard for reproducibility, reliability, and validity in temporal video understanding research (Zhang et al., 16 Dec 2025).

1. Concept and Historical Motivations

TimeLens-Bench is motivated by empirical revelations that legacy VTG benchmarks contain pervasive and systematic annotation errors, resulting in misleading model evaluations. Detailed audits showed, for example, that over one third of Charades-STA segment boundaries do not accurately delimit the queried event, over 20% of queries are duplicates within the same video (violating uniqueness), and roughly another quarter involve missing events, ambiguous queries, or temporally leaked information. These errors had the confounding effect that open-source models could appear to outperform advanced proprietary models—an artifact of data flaws rather than true capability. This prompted an overhaul of annotation standards and a disciplined reannotation initiative, operationalized as TimeLens-Bench (Zhang et al., 16 Dec 2025).

2. Annotation Criteria and Refinement Protocol

TimeLens-Bench enforces strict, reproducible annotation criteria:

• Query clarity & specificity: Each natural language query must be precise and unambiguous.
• Event existence: The referenced event must occur in the video.
• Query uniqueness: No two queries in the same video segment may describe the same event.
• No temporal leakage: Queries may not hint at absolute or relative timestamps.
• Temporal precision: Annotated segments must tightly enclose the event and nothing more.
• Annotation exhaustiveness: No interval outside the provided segment may satisfy the query semantics.

A two-stage “diagnose-then-refine” pipeline was deployed: expert annotators first reviewed video–query pairs to flag five error types (multiple occurrences, missing event, duplicate, unclear language, inaccurate timestamp), then—without reference to original timestamps—corrected identified samples by revising queries and reassigning temporal boundaries. Annotators were selected through trial annotations, provided a detailed handbook, and retained responsibility for both error detection and correction for consistency. Although no explicit inter-annotator agreement is reported, the protocol permits measurement via κ = (P̄ – P̄_e)/(1 – P̄_e), where P̄ is empirical agreement and P̄_e is chance (Zhang et al., 16 Dec 2025).

3. Composition and Dataset Statistics

TimeLens-Bench consists of three refined splits:

Split	#Videos	Avg. Duration (s)	#Annots.	#Queries Rewritten	#Segments Refined
Charades-TimeLens	1,313	29.6	3,363	2,467	896
ActivityNet-TimeLens	1,455	134.9	4,500	3,137	1,363
QVHighlights-TimeLens	1,511	149.6	1,541	859	682
Total (TimeLens-Bench)	4,279	107.8	9,404	6,463	2,941

Across all benchmarks, approximately 25% of annotations were either rewritten or discarded, reflecting pervasive flaws in the original sources. This rigorous curation process ensures a high signal-to-noise ratio for temporal grounding tasks and decisively removes confounding data artifacts (Zhang et al., 16 Dec 2025).

4. Benchmarking Metrics and Redefinition of Model Evaluation

TimeLens-Bench employs formalized VTG metrics:

• Intersection-over-Union (IoU):

$$\mathrm{IoU}(S_\mathrm{pred}, S_\mathrm{gt}) = \frac{|S_\mathrm{pred} \cap S_\mathrm{gt}|}{|S_\mathrm{pred} \cup S_\mathrm{gt}|}$$

• Mean IoU:

$$\mathrm{mIoU} = \frac{1}{N}\sum_{i=1}^{N}\mathrm{IoU}_i$$

• R1@m (Recall at threshold m):

$$\mathrm{R1}@m = \frac{1}{N}\sum_{i} [\,\mathrm{IoU}_i \geq m\,]$$

Analysis on TimeLens-Bench demonstrated dramatic model re-ranking compared to legacy datasets. On outdated versions, open-source systems such as VideoChat-Flash-7B could appear superior (e.g., mIoU≈45.2%) to GPT-5 (≈28.4%), a reversal of expectation; however, with the refined benchmarks, proprietary models correctly led (GPT-5 mIoU≈40.5%, VideoChat-Flash-7B≈39.7%). This also resulted in improved correlation between leaderboard rankings and actual model capacity (Zhang et al., 16 Dec 2025).

5. TimeLens-Bench Workflow and Toolkits

Three core resources support the adoption of TimeLens-Bench:

1. Open-source annotation toolkit: Including an error taxonomy, interface, and procedural handbook for internal data audits or curation.
2. Downloadable refined splits: The three TimeLens-Bench benchmark splits are directly usable for evaluation.
3. Automated re-annotation pipeline: For large-scale training sets (distinct from the small-scale, human-verified benchmarks), the authors built a system that leverages a state-of-the-art multimodal LLM (Gemini-2.5-Pro) to discover, query, timestamp, and self-verify candidate events, resulting in TimeLens-100K—a 20,000 video, 100,000 annotation training corpus absent of the gross errors that plague prior large-scale VTG datasets (Zhang et al., 16 Dec 2025).

6. Research Impact, Adoption, and Broader Context

TimeLens-Bench establishes a new standard for VTG research:

• It removes spurious noise that had misrepresented true model performance.
• It imposes rigorous, reproducible annotation criteria applicable to future benchmarks.
• The curation procedure and error taxonomy can be adapted by the broader community to audit or refine new video–text temporal datasets.
• The release of TimeLens-100K enables more effective training and immediate performance gains by eliminating data-driven confounds.

By aligning evaluation with reliable ground truth, TimeLens-Bench ensures that progress in VTG depends on substantive model advancements rather than artifacts of data pathology. This initiative is orthogonal to prior efforts in event-based frame interpolation "Time Lens++" (Tulyakov et al., 2022), temporal reasoning in LLMs "TimeBench" (Chu et al., 2023), and strong lens time-delay inference "Strong Lens Time Delay Challenge" (Liao et al., 2014), though it shares the principle that benchmarking must be firmly rooted in data validity and interpretability.

7. Implications for Future Dataset Construction and Evaluation

TimeLens-Bench exemplifies the necessity for continuous re-evaluation, auditability, and documentation in video temporal grounding benchmarks. The annotation protocols—clarity, existence, uniqueness, precision, no leakage, exhaustiveness—are broadly transferable to new domains and modalities. The dual approach of manual correction for small-scale reference sets and automated re-annotation for large-scale training sets informs future best practices for constructing high-fidelity multimodal datasets suitable for robust machine learning and model interpretability (Zhang et al., 16 Dec 2025).