Jamendo-QA: Audio Music QA Benchmark

• Jamendo-QA is a large-scale benchmark that integrates audio, metadata, and auto-generated QA pairs to support detailed music understanding.
• It employs an automatic annotation pipeline using Qwen-Omni to convert raw audio and metadata into nuanced questions and answers, emphasizing fine-grained temporal and spectral reasoning.
• The dataset addresses challenges such as genre and gender imbalances, serving as a foundational resource in the broader Jamendo music ecosystem for research and model evaluation.

Jamendo-QA is a large-scale Music Question Answering dataset built from freely licensed tracks on the Jamendo platform and designed as an audio-centered benchmark for natural-language question answering over music (Koh et al., 19 Sep 2025). It combines music audio, captions, metadata, and question-answer pairs through an automatic annotation pipeline based on Qwen-Omni, and is positioned as a publicly available resource for music understanding, music retrieval, generative modeling, and fair evaluation. In contrast to prior resources that are small, manually curated, audio-visual, or centered on symbolic music rather than raw audio, Jamendo-QA is intended to support fine-grained reasoning over temporal and spectral structure in music rather than only broad tagging or captioning (Koh et al., 19 Sep 2025).

1. Definition and research motivation

Jamendo-QA is introduced as a new benchmark for answering natural-language questions about music audio (Koh et al., 19 Sep 2025). Its central motivation is the shortage of large, diverse, music-specific QA data: prior datasets are described as often small, manually curated, limited to audio-visual settings, or centered on symbolic music rather than raw audio. The dataset is therefore framed as a scalable and publicly available resource intended to fill a gap in Music-QA research and to support music understanding, music retrieval, generative modeling, and fairer benchmarking across genres and metadata conditions (Koh et al., 19 Sep 2025).

A distinctive aspect of the benchmark is its emphasis on raw-audio reasoning. The paper states that Jamendo-QA is meant to help models learn fine-grained reasoning over temporal and spectral structure in music, rather than only performing coarse audio tagging or producing captions. This positions the dataset at the intersection of multimodal modeling and music information retrieval, where the target behavior is question answering grounded in audio content and aligned metadata rather than purely textual inference (Koh et al., 19 Sep 2025).

The paper also situates Jamendo-QA among adjacent music-language resources by scale and task type:

Dataset	Size	Scope
MUSIC-AVQA	45K	audio-visual QA
MusicQA	13K	music/audio QA
MTG-Jamendo	55K	tagging
JamendoMaxCaps	360K	captioning
LP-MusicCaps	542K	captioning
MusicXQA	1.29M	MIDI/sheet-based QA
Jamendo-QA	37K	audio QA

Within this comparison, Jamendo-QA is neither the largest Jamendo-derived resource nor a captioning corpus; its role is specifically to provide an audio QA benchmark grounded in freely licensed music and aligned textual supervision (Koh et al., 19 Sep 2025).

2. Source corpus and annotation pipeline

The dataset is built from freely licensed Jamendo tracks with available audio and metadata (Koh et al., 19 Sep 2025). Jamendo is treated as a strong foundation because it provides open-access music with rich metadata such as genre, instruments, artist info, speed/tempo, gender, length, and language. The paper does not describe a manual curation protocol with explicit inclusion or exclusion rules in the available description, but it makes clear that the final resource is assembled from Jamendo tracks with audio and metadata (Koh et al., 19 Sep 2025).

Its key technical mechanism is an automatic generation pipeline based on Qwen-Omni, referred to as a multimodal LLM. The workflow is described as follows: a raw music track is processed through an audio encoder to obtain audio embeddings that capture timbre, rhythm, harmony, and related information; optional metadata text such as genre, instruments, and artist information is provided; audio and text embeddings are then fed into Qwen-Omni; and the model generates music captions, questions about the music, and answers corresponding to those questions (Koh et al., 19 Sep 2025). The paper presents this as automatic annotation rather than manual labeling, enabling large-scale creation of aligned captions and QA pairs without the cost of human annotation.

The generated data are organized in a simple schema. Each sample contains an audio_path and a list of question-answer pairs in conversations. The paper also provides a minimal metadata example with fields such as genre, speed, gender, length_sec, and lang. The dataset and data card are stated to be publicly available on Hugging Face at https://huggingface.co/datasets/m-a-a-p/Jamendo-QA (Koh et al., 19 Sep 2025).

3. Scale, composition, and descriptive statistics

The principal scale figure reported for Jamendo-QA is 37K examples (Koh et al., 19 Sep 2025). The paper describes the dataset as covering a diverse range of genres, instruments, and metadata attributes, although it does not provide explicit counts for the number of captions, distinct genres, distinct instruments, or total metadata attributes in the available description.

The composition is explicitly non-uniform. The analysis states that the dataset is dominated by rock and pop, followed by guitar and indie. The metadata space is also described as showing meaningful separability across dimensions including genre, speed, gender, duration, and language (Koh et al., 19 Sep 2025). This indicates that the dataset is broad enough to expose structured metadata variation, while also exhibiting concentration around a few dominant musical categories.

The paper reports several audio statistics:

Statistic	Mean	Median	Mode
Length (sec)	233.4	220.5	240.0
SNR (dB)	12.48	11.69	9.38
RMS Energy	0.195	0.191	0.120
Zero Crossing Rate	0.057	0.054	0.038

These values indicate that the tracks are mostly standard-length songs, typically around 3–4 minutes (Koh et al., 19 Sep 2025). No explicit train, validation, or test split is described in the paper details provided. Likewise, the paper emphasizes dataset construction and analysis rather than a formal task protocol with fixed benchmark partitions.

4. Question coverage and task characteristics

Jamendo-QA is described as providing balanced coverage across four major QA categories: genre, speed, duration, and gender (Koh et al., 19 Sep 2025). This categorical organization is one of the paper’s central claims, because it means the benchmark is not restricted to a single style of music question but instead spans multiple metadata-grounded dimensions of music understanding.

Representative question types include “What is the dominant instrument?” and “What is the mood of this song?” (Koh et al., 19 Sep 2025). The metadata example further implies questions and answers tied to genre identification, tempo or speed classification, artist gender, and song length. The paper’s discussion suggests that the generated QA pairs are mostly factual and metadata-grounded rather than open-ended subjective interpretation. This suggests a benchmark design oriented more toward controlled semantic grounding than toward unconstrained aesthetic commentary (Koh et al., 19 Sep 2025).

The paper is primarily a dataset paper rather than a benchmark-results paper. It mentions related model families in the introduction, including MU-LLaMA, MuMu-LLaMA, CLAP, MERT, and Qwen-Omni as the annotation backbone, but it does not report detailed experimental benchmarks, supervised training baselines, or zero-shot evaluation results for Jamendo-QA itself (Koh et al., 19 Sep 2025). Similarly, no complex mathematical objective or task-specific evaluation metric is defined in the available description. The formal content is largely descriptive, with the closest explicit formalization being the dataset schema centered on audio_path and conversations.

5. Biases, limitations, and fairness considerations

The paper explicitly documents several biases and limitations in Jamendo-QA (Koh et al., 19 Sep 2025). The most prominent is genre imbalance: rock and pop, along with guitar-related tracks, dominate all four question types. This creates a potential bias toward rock and pop music and may make underrepresented genres harder for models to generalize to.

A second major issue is gender imbalance. The QA distribution is described as male-vocal skewed, and this skew is visible across question types. The paper presents this as a fairness concern for downstream applications and for model evaluation, since performance on a male-skewed corpus may not reflect robust behavior across vocal categories (Koh et al., 19 Sep 2025).

A third imbalance concerns tempo. The dataset is biased toward medium tempo tracks, which dominate the QA generation process. Together with the genre and gender skews, this means the benchmark is diverse but not distributionally balanced. Any benchmark results obtained on Jamendo-QA therefore need to be interpreted with attention to uneven coverage across metadata conditions (Koh et al., 19 Sep 2025).

The annotation process introduces an additional limitation. Because questions, answers, and captions are automatically generated by Qwen-Omni, the dataset may inherit generator biases or hallucination risks from the model. The paper does not provide a deep analysis of annotation errors in the available description, but the fully automated pipeline is itself identified as a limitation relative to human-validated labels (Koh et al., 19 Sep 2025). A plausible implication is that Jamendo-QA should be viewed simultaneously as a large-scale supervision resource and as a benchmark whose label quality is conditioned by automatic generation.

6. Position within the Jamendo research ecosystem

Jamendo-QA occupies the role of a single-track QA foundation within a broader Jamendo-centered family of music-language resources. Its most direct extension is Jamendo-MT-QA, a multi-track comparative benchmark built on top of Jamendo-QA (Koh et al., 8 Apr 2026). The relationship between the two is described as direct: Jamendo-QA provides the original single-track foundation, with track-level audio, metadata, captions, and QA pairs, while Jamendo-MT-QA repurposes that resource into paired-track reasoning tasks. Jamendo-MT-QA contains 36,519 comparative QA items over 12,173 track pairs, with each pair yielding exactly three question types: yes/no, short-answer, and sentence-level (Koh et al., 8 Apr 2026). This extension shows that Jamendo-QA is not only a standalone benchmark but also a substrate for more complex relational and comparative music reasoning.

A second adjacent resource is JamendoMaxCaps, which is not itself a QA benchmark but is highly relevant as a Jamendo-derived caption and metadata resource (Roy et al., 11 Feb 2025). JamendoMaxCaps contains 362,238 instrumental tracks after removing very short tracks, uses Qwen2-Audio to generate captions, and augments partial metadata through retrieval plus a locally hosted Llama-2 7B model (Roy et al., 11 Feb 2025). Its relevance to Jamendo-QA lies in providing a large captioned Jamendo corpus with richer metadata, which can support retrieval-augmented QA, caption-based QA, and music-language representation learning.

Taken together, these works place Jamendo-QA in a layered research ecosystem. JamendoMaxCaps expands the descriptive and metadata-rich Jamendo substrate; Jamendo-QA turns Jamendo audio into a 37K-scale audio QA benchmark; and Jamendo-MT-QA extends the single-track formulation into comparative, multi-track reasoning (Roy et al., 11 Feb 2025, Koh et al., 8 Apr 2026). This suggests that Jamendo-QA functions as a foundational benchmark for audio-grounded music question answering, while also serving as an intermediate representation layer from which more elaborate music-language tasks can be constructed.