LongAudio-XL: Long-Audio QA Dataset

• LongAudio-XL is a large-scale dataset designed for long-context audio reasoning, featuring over 1M QA examples from speech, sound, and music spanning 30 seconds to 10 minutes.
• It integrates diverse sources like podcasts, debates, and single-speaker audiobooks to support tasks such as captioning, temporal, needle, and subscene QA.
• Its integration into Audio Flamingo training through staged, LoRA-based methods enhances model performance on long-audio benchmarks and reasoning tasks.

LongAudio-XL denotes the long-audio component of the Audio Flamingo data suite. In Audio Flamingo 3, it is introduced as a large-scale long-audio question-answering dataset that extends the earlier LongAudio resource from sounds and music to long-form speech by incorporating over 1M reasoning-focused QA examples from long-form speech spanning 30 seconds to 10 minutes; in Audio Flamingo Next, the term becomes an umbrella for an expanded family of long-audio captioning and QA corpora built from real-world 5–30-minute internet audio and integrated with temporal reasoning data (Goel et al., 10 Jul 2025, Ghosh et al., 13 Apr 2026).

1. Genealogy and conceptual scope

The immediate precursor to LongAudio-XL is LongAudio from Audio Flamingo 2. That earlier dataset is described as “the first large-scale long audio understanding dataset,” comprising over 80K unique audios and 262,928 QA pairs, with a stated design emphasis on audios ranging from 30 seconds to 5 minutes and an average duration of 117.08 seconds. Its task inventory already included captioning, plot QA, temporal QA, needle QA, subscene QA, and general QA, and it was created primarily for non-speech audio and music (Ghosh et al., 6 Mar 2025).

Audio Flamingo 3 redefines the problem space by stating that the existing LongAudio dataset was focused on sounds and music and by extending it with over 1M reasoning-focused QA examples from long-form speech. In that formulation, LongAudio-XL is a large-scale long-audio QA resource designed to supervise long-context reasoning over speech, sound, and music, with particular emphasis on audio of at least 30 seconds and up to approximately 10 minutes (Goel et al., 10 Jul 2025).

Audio Flamingo Next preserves the name but changes its operational meaning. The paper explicitly says that it expands the existing AudioSkills-XL, LongAudio-XL, AF-Think, and AF-Chat datasets, yet it does not define LongAudio-XL in a single standalone section. Instead, LongAudio-XL is realized through several long-audio datasets—Long Audio Captioning, Long Temporal QA, Long Needle QA, Long Subscene QA, and Long Counting QA—built from approximately 200K long videos from the open internet, spanning durations of up to 5 to 30 minutes (Ghosh et al., 13 Apr 2026).

A frequent source of confusion is that LongAudio-XL is not a single immutable benchmark. In AF3 it is primarily an expanded long-audio QA suite centered on long-form speech; in AF-Next it is a broader long-audio training layer coupled to time-grounded reasoning and multi-turn instruction data. Another important distinction is that LongAudio-XL is a supervision resource for long-audio understanding, not a raw-waveform generative model. Raw-audio million-context modeling appears in adjacent work on autoregressive waveform prediction, where long context is defined in raw samples rather than QA supervision (Verma, 2022).

2. Dataset composition and task taxonomy

In AF3, the defining extension is the addition of long-form speech. The source corpora include LibriSpeech, EuroParl, VoxPopuli, Spotify Podcasts, Switchboard, Fisher, MELD, DailyTalk, MMDialog, and IEMOCAP. These corpora supply single-speaker speeches, audiobooks, parliamentary debates, telephone dialogues, emotional dyadic conversations, multi-party dialogues, and podcasts, giving LongAudio-XL a discourse range that spans monologic, dialogic, and broadcast-style speech (Goel et al., 10 Jul 2025).

The AF3 task taxonomy is explicitly reasoning-oriented. It includes sarcasm identification; emotional state reasoning, with identification, causal reasoning, and emotion flip; topic relationship reasoning; information extraction, with needle QA, causal QA, response QA, and topic QA; summarization; and order or temporal reasoning, including temporal order, temporal attribute, temporal referring, and temporal grounding. The supervision is predominantly multiple-choice, single-turn, and aligned to a single long audio clip per example (Goel et al., 10 Jul 2025).

The AF3 appendix tables also show that the long-speech component is large and heterogeneous at the per-skill level. For example, summarization includes 103,920 Spotify Podcasts instances with an average duration of 2002.99 seconds and a maximum duration of 18,206.44 seconds, while needle-style information extraction includes another 103,920 Spotify Podcasts instances at the same average duration. This indicates that LongAudio-XL was constructed from sources whose native durations substantially exceed conventional short-clip audio benchmarks, even though AF3 frames its long-speech reasoning target as 30 seconds to 10 minutes (Goel et al., 10 Jul 2025).

AF-Next restructures the long-audio portion into five named long-audio datasets. Their reported scales are 27k hours and 290k pairs for Long Audio Captioning, 39k hours and 317k pairs for Long Temporal QA, 34k hours and 281k pairs for Long Needle QA, 28k hours and 256k pairs for Long Subscene QA, and 26k hours and 170k pairs for Long Counting QA. Together these datasets amount to roughly 154k hours of long-audio supervision and form the concrete substrate of LongAudio-XL in the AF-Next regime (Ghosh et al., 13 Apr 2026).

These AF-Next long-audio datasets shift the emphasis from concatenated academic corpora toward real-world, naturally long recordings. The paper characterizes them as being derived from more than approximately 200K long videos, with audio types that include speech-heavy media, mixed soundtracks, and music, and with examples ranging from 5 to 30 minutes. The long-audio tasks themselves emphasize temporal understanding, needle-in-the-haystack retrieval, localized subscene reasoning, and counting over long spans (Ghosh et al., 13 Apr 2026).

3. Curation and annotation pipeline

The AF3 construction pipeline is transcript-centric and chronologically grounded. The paper states that it merges consecutive short segments in chronological order to construct longer, coherent audios. It also uses LLM prompting to select information-dense segments and to synthesize reasoning-focused QA pairs from transcripts, speaker information, and, where available, emotion labels. GPT-4.1 and Gemini 2.0 Flash are the named generators for these long-speech questions (Goel et al., 10 Jul 2025).

The AF3 prompts enforce a structured reasoning style rather than surface-form retrieval. Topic Relationship prompts ask the generator to read the full conversation or speech and create a question about how two topics or ideas are related; Needle QA prompts require locating a specific detail that “requires reading the entire transcript to locate”; Order QA prompts require reconstructing the order of events or topics; causal and emotional prompts emphasize context-sensitive explanation rather than lexical matching. The resulting dataset is therefore synthetic in annotation origin but explicitly optimized for long-context discourse reasoning (Goel et al., 10 Jul 2025).

The released supervision in AF3 is not the raw transcript. The paper states that transcripts are used to build questions, but the transcripts themselves are not part of the released QA supervision; they are an intermediate artifact. It also states that LongAudio-XL is single-turn per audio, while multi-turn dialogue is handled separately by AF-Chat. Similarly, chain-of-thought is not native to the base LongAudio-XL examples: a subset is later sampled into AF-Think, where short reasoning prefixes are synthesized with the ground-truth answer supplied (Goel et al., 10 Jul 2025).

AF-Next adopts a different acquisition pipeline oriented around genuine long internet media. It collects approximately 200K long videos through agentic web search, then produces 10-second segment-level annotations consisting of video captions, audio-only captions, speech transcripts, and paralinguistic descriptions. An LLM is then prompted to merge these segment annotations into a coherent long caption for the entire audio. Further prompts synthesize Long Temporal QA, Long Needle QA, Long Subscene QA, and Long Counting QA from these caption and transcript traces (Ghosh et al., 13 Apr 2026).

The AF-Next paper also records explicit quality-control constraints. It states that it limits synthesized QA types for long audio to settings where the current pipeline is robust, because other QA types were found to be more prone to hallucination. It further describes filtering short extracted segments by informativeness and notes anonymization and privacy constraints when long-audio sources might contain personal voices. This suggests a curation philosophy in which scale is coupled to aggressive synthetic filtering rather than unrestricted QA generation (Ghosh et al., 13 Apr 2026).

4. Integration into Audio Flamingo training

In AF3, LongAudio-XL is central to Stage 3.5, titled “Context Extension and Thinking.” At this stage the model incorporates LongAudio-XL and AF-Think and adopts LoRA-based training by freezing the model’s original weights and training LoRA adapters for the LLM. The optimization settings reported for this stage are per-device batch size 4, learning rate $5 \times 10^{-5}$, 2 epochs, and gradient accumulation 4, with bf16 mixed precision and ZeRO-3 distributed training (Goel et al., 10 Jul 2025).

The long-audio front end in AF3 is AF-Whisper. Audio is resampled to 16 kHz mono, converted to a 128-channel mel-spectrogram with a 25 ms window and 10 ms hop, and processed by AF-Whisper, which outputs features at 50 Hz. Audio is split into non-overlapping 30-second windows, and a pooling layer with stride 2 is applied before the features are passed through an audio adaptor and into Qwen-2.5-7B. The maximum number of windows increases with training stage, and Stage 3.5 supports audio up to approximately 10 minutes (Goel et al., 10 Jul 2025).

The AF3 training logic is explicitly curricular. Stages 1 and 2 emphasize recognition and short-audio data with long context disabled; Stage 3 uses AudioSkills-XL to teach short- to medium-range skills; Stage 3.5 adds LongAudio-XL and AF-Think to extend those skills to long audio and to reasoning over long-context prefixes. A plausible implication is that LongAudio-XL functions less as a generic corpus than as the stage-specific mechanism that forces the model to operationalize long-context audio reasoning (Goel et al., 10 Jul 2025).

AF-Next moves long-audio from a late-stage extension into a major mid-training regime. During mid-training Stage 2, the model adds Long Audio Captioning, Long Temporal QA, Long Needle QA, Long Subscene QA, and Long Counting QA, increases the maximum audio length to 30 minutes, and extends the context length to 128k tokens. The paper states that the Stage-1 mixture is down-sampled to half of its original blend weights, while all long-audio datasets are assigned a blend weight of 1. The resulting checkpoint is called AF-Next-Captioner (Ghosh et al., 13 Apr 2026).

The AF-Next architecture supports this expansion with AF-Whisper features at 50 Hz, stride-2 pooling to an effective 25 Hz audio token rate, non-overlapping 30-second chunks, and a Qwen-2.5-7B backbone extended from 32k to 128k context. Temporal alignment is reinforced with RoTE, which replaces standard discrete-index rotation with absolute time-based rotation, $\theta \gets -\tau_i \cdot 2\pi$. LongAudio-XL is therefore tied not only to data scale but also to an explicit long-context positional scheme (Ghosh et al., 13 Apr 2026).

5. Empirical role and benchmark impact

The clearest empirical evidence for LongAudio-XL in AF3 comes from long-audio evaluation. On LongAudioBench, AF3 scores 68.6, compared with 60.4 for Gemini 2.5 Pro. On LongAudioBench + Speech, described as an extended version with approximately 2.5k human-annotated long speech QA instances, AF3 scores 72.9 against 66.2 for Gemini 2.5 Pro. The paper does not report a direct “with versus without LongAudio-XL” ablation, but it explicitly presents LongAudio-XL as the training resource responsible for long speech reasoning and uses these results as evidence of the capability (Goel et al., 10 Jul 2025).

AF-Next reports a further increase on the same long-audio benchmarks. On LongAudioBench, AF-Next-Instruct reaches 73.9, and on LongAudioBench + Speech it reaches 81.2. The same paper also reports a LibriSQA score of 9.3 for AF-Next-Instruct, compared with 8.7 for AF3. Because AF-Next modifies both the long-audio data mixture and the training/positional machinery, these gains cannot be attributed to LongAudio-XL alone, but they are the main benchmark signature of the expanded long-audio stack (Ghosh et al., 13 Apr 2026).

The earlier AF2 results help clarify why explicit long-audio finetuning matters. AF2 showed that adding LongAudio finetuning lifted LongAudioBench performance from 50.22% to 64.19%, with especially large gains on Needle QA and Subscene QA. This earlier transition from short-audio competence to long-audio competence is the lineage that AF3 converts into LongAudio-XL for speech and that AF-Next later extends to real-world 30-minute audio (Ghosh et al., 6 Mar 2025).

A broader reading across contemporary long-audio benchmarks indicates why such datasets matter. AudioMarathon defines long-context audio as 90–300-second clips and reports clear performance drops as audio length grows, especially on tasks requiring temporal reasoning and speaker-centric inference. ChronosAudio, which stratifies clips from 30 seconds to 20 minutes, reports “Precipitous Long-Context Collapse,” with over 90% degradation in specific tasks as models move from short to long contexts. These benchmarks were not built from LongAudio-XL, but they help explain the problem setting that LongAudio-XL is trying to regularize during training (He et al., 8 Oct 2025, Luo et al., 8 Jan 2026).

6. Relation to adjacent long-audio methods and persistent limitations

LongAudio-XL belongs to the data-centric branch of long-audio research. Other work addresses the same regime through architectural or systems interventions: Partial YaRN and Virtual Longform Audio Training extend audio context in RoPE-based LALMs by modifying only audio token positions; Speech-XL introduces Speech Summarization Tokens to retain only compressed KV proxies for speech intervals; GigaChat Audio interleaves periodic time markers with continuous audio tokens and reports timestamped reasoning over up to 120 minutes. These methods target long-form inference directly, whereas LongAudio-XL provides the training supervision needed for models to exploit such mechanisms on reasoning tasks (Chaichana et al., 17 Oct 2025, Sun et al., 5 Feb 2026, Kutsakov et al., 11 Jul 2026).

One misconception is that LongAudio-XL itself provides explicit chain-of-thought. AF3 states that the base dataset is primarily long-audio plus QA supervision and that CoT arises through AF-Think. AF-Next makes the separation even sharper by introducing AF-Think-Time for Temporal Audio Chain-of-Thought, where intermediate reasoning steps are explicitly grounded to timestamps. LongAudio-XL therefore supplies long-context tasks and contexts; timestamped reasoning traces belong to the adjacent “think” datasets (Goel et al., 10 Jul 2025, Ghosh et al., 13 Apr 2026).

Another misconception is that LongAudio-XL is a benchmark. In the Audio Flamingo papers it is primarily a training resource. LongAudioBench and LongAudioBench + Speech are the corresponding evaluation instruments, while broader external evaluations are provided by benchmarks such as AudioMarathon and ChronosAudio (Goel et al., 10 Jul 2025, He et al., 8 Oct 2025, Luo et al., 8 Jan 2026).

The limitations stated in AF3 are substantial. The system is English-only; LongAudio-XL relies heavily on GPT-4.1 and Gemini 2.0 Flash for synthetic data generation; and its domain mix is dominated by formal speech, phone conversations, and English podcasts. The paper also notes that base LongAudio-XL lacks native human-written chain-of-thought and that richer rationales remain future work (Goel et al., 10 Jul 2025).

AF-Next inherits several of these issues while expanding scale. Its long-audio data are drawn largely from English-dominant web media; its long-audio labels are synthesized from captioning and transcript pipelines; and the release is under a research-only license because of data licensing constraints. The paper also acknowledges that, despite timestamped CoT and 128k-context training, long-range reasoning across 30 minutes remains difficult, especially when evidence is sparse or widely distributed (Ghosh et al., 13 Apr 2026).

Taken together, these properties place LongAudio-XL at the center of a specific historical transition in audio-language modeling: from short-clip audio QA toward document-level audio reasoning. Its significance lies less in a single fixed data release than in the way it operationalizes long-context supervision across speech, sound, and music, and in how later Audio Flamingo systems build increasingly explicit temporal machinery on top of that foundation (Goel et al., 10 Jul 2025, Ghosh et al., 13 Apr 2026).