Listening with Time: Precise Temporal Awareness for Long-Form Audio Understanding

Abstract: While Large Audio LLMs (LALMs) achieve strong performance on short audio, they degrade on long-form inputs. This degradation is more severe in temporal awareness tasks, where temporal alignment becomes increasingly inaccurate as audio duration grows. We attribute these limitations to the lack of data, benchmarks, and modeling approaches tailored for long-form temporal awareness. To bridge this gap, we first construct LAT-Chronicle, a 1.2k hour long-form audio dataset with temporal annotations across real-world scenarios. We further develop LAT-Bench, the first human-verified benchmark supporting audio up to 30 minutes while covering three core tasks: Dense Audio Caption, Temporal Audio Grounding, and Targeted Audio Caption. Leveraging these resources, we propose LAT-Audio, formulating temporal awareness as a progressive global-to-local reasoning paradigm. A global timeline is first constructed as an aligned temporal-semantic context,and the Think-With-Audio Chain-of-Thought (TWA-CoT) is then introduced to perform iterative reasoning by incorporating local audio information via tool use. Experiments show that LAT-Audio surpasses existing models on long-form audio temporal awareness tasks and improves robustness to input duration. We release the dataset, benchmark, and model to facilitate future research at https://github.com/alanshaoTT/LAT-Audio-Repo.

• The paper introduces LAT-Audio, a progressive global-to-local temporal reasoning framework that mitigates hallucination and timestamp drift in long-form audio.
• It presents comprehensive datasets and benchmarks, LAT-Chronicle and LAT-Bench, featuring over 1,200 hours of annotated audio for diverse tasks.
• Empirical results show significant improvements over existing methods with strong performance metrics, underscoring the model’s robustness in complex scenarios.

Precise Temporal Awareness for Long-Form Audio: LAT-Audio and Its Benchmarking Infrastructure

Motivation and Problem Characterization

The paper "Listening with Time: Precise Temporal Awareness for Long-Form Audio Understanding" (2604.22245) addresses a fundamental gap in Large Audio LLM (LALM) research: robust and precise temporal awareness on long-form audio, particularly in real-world settings where audio durations extend well beyond conventional short clips (e.g., podcasts, meetings, broadcasts). Conventional LALMs exhibit two prominent temporal alignment failures: hallucination (predicted events falling outside the true temporal range) and timestamp drift (cumulative misalignment over time), especially in tasks requiring event localization or temporal grounding. These errors become more pronounced as audio duration and acoustic complexity increase.

Figure 1: Typical temporal hallucination and timestamp drift failures in LATA tasks.

Dataset and Benchmark Construction: LAT-Chronicle & LAT-Bench

To foster systematic progress in temporal-aware audio modeling, the paper introduces two foundational resources: LAT-Chronicle and LAT-Bench. LAT-Chronicle is a 1,200-hour long-form audio dataset, annotated with temporally grounded labels for Dense Audio Captioning (DAC), Temporal Audio Grounding (TAG), and Targeted Audio Captioning (TAC). Task annotation employs a multi-track, human-in-the-loop pipeline (LAT-Pipe) which combines automatic track decomposition (speech, sound, music, environment) and forced alignment with manual validation to establish high temporal precision across six real-world scenario types and two languages.

Figure 2: The multi-stage LAT-Pipe annotation pipeline, central to dataset construction.

LAT-Bench is a human-verified benchmark consisting of 40 hours of long-form audio, curated for evaluation in DAC, TAG, and TAC, supporting audio lengths up to 30 minutes and embracing scenario and language diversity.

Figure 3: Duration and scenario distributions for both LAT-Chronicle and LAT-Bench.

LAT-Audio: Progressive Global-to-Local Temporal Reasoning

The paper proposes LAT-Audio, a temporal reasoning framework that achieves robust performance on long-form audio tasks by progressive global-to-local segmentation. The model first generates a global timeline, segmenting the audio into coarse but semantically aligned spans, thus enabling temporally structured context. Subsequently, for each task, LAT-Audio performs iterative chain-of-thought (TWA-CoT) reasoning: at each step, the model may use a ‘tool call’ to crop local audio for fine-grained analysis, leveraging both the global timeline as context and local segment features for correction and progressive refinement.

Figure 4: LAT-Audio framework: global timeline construction (left) and progressive global-to-local reasoning with TWA-CoT (right).

The approach mitigates attention dilution and positional encoding limits inherent to vanilla LLMs facing long input sequences, while resisting loss of contextual continuity that plagues sliding-window baselines.

Training and Optimization Methodology

Model training is split into three sequential stages:

• Stage 1: Supervised fine-tuning (SFT) for global timeline prediction using annotated outputs.
• Stage 2: SFT for full reasoning trajectory generation via oracle-conditioned LLM traces.
• Stage 3: Group Relative Policy Optimization (GRPO)-based reinforcement learning, leveraging a mixture of correct and incorrect sample trajectories to strengthen temporal reasoning robustness.

Context management relies on selective temporal downsampling to alleviate the token budget run-up while retaining fine local detail at subsequent reasoning steps.

Numerical Evaluation and Ablation

LAT-Audio demonstrates strong task scores on LAT-Bench: TAG mIoU 47.2/50.0 (ZH/EN), DAC avg 46.8/48.6, TAC FENSE 62.0/68.7. On BLAB advertisement localization (for up to 30min audio), LAT-Audio achieves mIoU 49.3, [email protected] 66.7. These results surpass Gemini-2.5-Pro and Gemini-3.0-Pro by large margins, especially in long-duration contexts, where baseline models incur sharp drops (e.g., Gemini-2.5-Pro TAG falls from 62.6 to 16.1 beyond 15 minutes). Sliding-window approaches deteriorate global context and temporal continuity, harming accuracy for temporal grounding and dense captioning.

Performance degrades gracefully for LAT-Audio as audio duration or scenario complexity (particularly high-overlap contexts) increases, evidencing superior robustness compared to both direct long-context LALMs and chunk-based methods.

Figure 5: Comparative model robustness across increasing audio durations and scenario complexity.

Ablations show that removing global timeline, TWA-CoT, or reinforcement learning leads to sharp drops; aggressive temporal downsampling further deteriorates results beyond a 2× setting, confirming necessity of the progressive global-to-local paradigm and audio-grounded iterative reasoning.

Practical and Theoretical Implications

The research establishes that precise temporal reasoning in audio LMs requires explicit temporal structuring and iterative evidence retrieval, rather than monolithic token-level attention or chunking. Task formulation as global-to-local reasoning generalizes across captioning and localization. The dataset and benchmark open avenues for multilingual, acoustically complex, and temporally-aware audio modeling, supporting not only transcription and event detection but context-rich, narrative audio understanding.

Practically, this enables next-generation audio systems for meeting summarization, podcast analysis, and agentic listening in extended interactions, with guaranteed alignment and reduced hallucination. The demonstrated robustness may generalize to multimodal, multi-event, and long-context agentic tasks as tool-use paradigms mature.

Conclusion

The paper delivers a comprehensive solution for long-form audio temporal awareness through robust benchmark construction and a progressive global-to-local reasoning architecture. The strong empirical results, coupled with rigorous analysis of failure modes and robustness, establish both the necessity and efficacy of structured temporal modeling in LALMs. Future development will need to address computational overhead intrinsic to iterative tool calls, expand into cross-modal temporal reasoning, and further scale annotation and evaluation frameworks for broader agentic audio understanding.