TimePro-RL: Temporal Alignment for Audio Models

• The paper introduces TimePro-RL, a framework that integrates explicit timestamp embeddings with reinforcement learning to achieve fine-grained temporal accuracy in audio event localization.
• It leverages the Audio-Side Time Prompt (ASTP) to interleave high-resolution timestamp tokens with audio features, enhancing performance on tasks like Audio Grounding, SED, and Dense Audio Captioning.
• Ablation studies confirm that semantic timestamp initialization and adaptive reward fusion are critical for optimizing performance without compromising language quality.

TimePro-RL is a post-training framework designed to enable Large Audio-LLMs (LALMs) to achieve fine-grained temporal perception by explicitly modeling timestamp coordinates and directly optimizing temporal alignment. The framework introduces the Audio-Side Time Prompt (ASTP)—which interleaves high-resolution timestamp embeddings with audio frame features—and leverages Reinforcement Learning (RL) to align output event boundaries with audio content across tasks such as Audio Grounding (AG), Sound Event Detection (SED), and Dense Audio Captioning (DAC) (Shi et al., 15 Apr 2026).

1. Motivation and Problem Definition

LALMs such as Qwen2-Audio typically couple an audio encoder (e.g., Whisper) with an autoregressive LLM, enabling general semantic understanding but lacking mechanisms for precise temporal inference. Conventional positional encodings (e.g., RoPE) capture only relative token order, failing to encode absolute wall-clock time required in temporal localization tasks. Consequently, existing LALMs can respond to “What sound occurs at 10 seconds?” by generating event descriptions without reliably attaching precise onset or offset timestamps.

Tasks demanding high temporal fidelity—such as AG (mapping text or query to specific audio intervals), SED (detecting sound event boundaries with sub-second accuracy), and DAC (aligning audio captions with precise events)—expose LALM limitations. Furthermore, standard supervised fine-tuning (SFT), grounded in token-level cross-entropy, heavily penalizes minor temporal deviations despite substantial overlap, leading to misaligned optimization dynamics detrimental to temporal tasks (Shi et al., 15 Apr 2026).

2. Framework Architecture and Data Encoding

TimePro-RL builds on a two-stage pipeline: ASTP injection followed by SFT and RL-driven optimization.

2.1 Audio-Side Time Prompt (ASTP)

The ASTP mechanism extends the text tokenizer to include a vocabulary of timestamp tokens,

$$T = \{\langle 0.00\rangle, \langle 0.04\rangle, \langle 0.08\rangle, \ldots, \langle 30.00\rangle\}$$

spaced at Δ = 0.04 s to match the 25 Hz frame rate of Whisper. For an audio clip of $L$ seconds, the preprocessing step yields $J = 25 \cdot L$ frame features $\{a_1, \ldots, a_J\}$. Each frame $j$ is paired with a timestamp $\tau_j = (j-1)/25$ s, and the corresponding token $\langle\tau_j\rangle$ is inserted before $a_j$, yielding the interleaved sequence

$$x = [\ldots, t_i, a_j, t_{i+1}, a_{j+1}, \ldots] \in \mathbb{R}^{(2J)\times d}$$

where $t_i = f_{emb}(\tau_i)$ denotes the embedding of timestamp $L$0 produced by a learned but frozen embedding layer.

Timestamp embeddings are initialized via averaging the pretrained embeddings of their subword tokenization:

$L$1

All timestamp embeddings remain frozen thereafter, preserving their semantic anchoring.

2.2 Training Protocol

After ASTP injection, standard SFT is carried out for three epochs ($L$2, LoRA $L$3, $L$4), using token-level cross-entropy. Although this step enhances basic timestamp awareness, SFT alone remains suboptimal for fine boundary localization due to misalignment between loss and evaluation metrics (Shi et al., 15 Apr 2026).

3. Reinforcement Learning Optimization

Recognizing SFT's limitations for boundary-sensitive prediction, TimePro-RL introduces policy-gradient RL using Group Relative Policy Optimization (GRPO).

• Reward construction:
  • Main reward $L$5: Event-based F1 (Eb-F1) reflecting correctly predicted event boundaries within a prespecified tolerance.
  • Auxiliary reward $L$6: Smoother, task-specific overlap metric (mean IoU for AG/SED; METEOR score for DAC).
• Adaptive reward fusion: For each group of $L$7 sampled predictions, the reward vector $L$8 is set as

$L$9

with $J = 25 \cdot L$0. If group Eb-F1 values are constant (low variance), the auxiliary reward modulates the signal to provide smoother gradients.

• Policy objective:

$J = 25 \cdot L$1

Updates use the group-relative advantage $J = 25 \cdot L$2:

$J = 25 \cdot L$3

Training proceeds for one RL epoch on 10,200 examples at learning rate $J = 25 \cdot L$4 (Shi et al., 15 Apr 2026).

4. Experimental Protocols and Performance

Experiments evaluate TimePro-RL on three temporally sensitive audio-language tasks:

Task	Data Source	Evaluation Metrics	Train/Test Samples
Audio Grounding	FTAR	[email protected]/0.7/0.9, mIoU	61,862 / 483
Sound Event Detection	DESED	Eb-F1 (±0.2 s tolerance)	15,041 / 1,153
Dense Audio Captioning	FTAR	METEOR (lang), Eb-F1 (temporal)	92,443 / 741

Key results (improvement over SFT/Qwen2-Audio baselines):

• Qwen2-Audio (7B) post-SFT: [email protected] ≈ 75, Eb-F1(SED) ≈ 49, METEOR ≈ 32, Eb-F1(DAC) ≈ 35.
• TimePro-RL:
  • [email protected] = 78.8 (+4.0), [email protected] = 38.1 (+3.5)
  • Eb-F1(SED) = 58.4 (+8.6)
  • METEOR(DAC) = 35.3 (+3.1), Eb-F1(DAC) = 39.8 (+4.8)
• Qwen2.5-Omni (7B) post-TimePro-RL: Similar or slightly stronger gains (Shi et al., 15 Apr 2026).

5. Ablation Studies

Ablation results highlight the importance of timestamp initialization and reward design:

• ASTP with random (non-semantic) initialization degrades performance across all tasks, indicating sensitivity to embedding semantics.
• ASTP with semantic initialization alone confers modest improvements (e.g., +1.7 points in AG [email protected], +1.2 in Eb-F1(SED)).
• Addition of RL with Eb-F1 alone as reward further improves boundary-sensitive metrics but can degrade language quality (e.g. METEOR for captions).
• Using the adaptive fused reward restores METEOR and achieves best overall task gains (e.g., METEOR +1.3, Eb-F1(DAC) +3.7), suggesting that reward smoothing is critical to prevent degradation of text-level output quality (Shi et al., 15 Apr 2026).

6. Limitations and Prospective Extensions

TimePro-RL currently supports only single-turn, timestamp-anchored queries. Extending to multi-step or temporal chain-of-thought reasoning is suggested as future work. The timestamp vocabulary is fixed (0–30 s, 0.04 s stride); processing longer audio or requiring variable temporal granularity would demand adaptive prompt construction. RL post-training operates on a restricted data subset (10,200 examples, one epoch), potentially limiting sample efficiency. Proposed directions include exploring dynamic timestamp tokenization, more effective advantage estimation, and human feedback-based reward models (Shi et al., 15 Apr 2026).

7. Broader Implications

TimePro-RL establishes a paradigm for augmenting LALMs with explicit time grounding, addressing a critical bottleneck in generalized audio language modeling. Its architectural modularity (independent of audio backbone) facilitates adoption across diverse LALMs, and its reinforcement-learning formulation directly optimizes the alignment between audio events and predicted timestamps. A plausible implication is that explicit temporal conditioning, coupled with alignment-metric rewards, forms a necessary ingredient for deploying LALMs in structured audio event understanding applications such as multimodal video understanding, surveillance, and auditory scene analysis.