COCOLA: Contrastive Learning for Musical Coherence

• COCOLA is a contrastive learning method that captures harmonic and rhythmic coherence in musical audio through sub-mix embeddings.
• It uses a bilinear similarity function with a stem-level contrastive objective to assess the compatibility between conditioning and generated audio.
• Empirical results show high accuracy in detecting coherent sub-mixes, aligning well with qualitative evaluations in accompaniment generation.

Searching arXiv for COCOLA and closely related papers to ground the article. arxiv_search(query="COCOLA coherence-oriented contrastive learning musical audio representations", max_results=10, sort_by="relevance") COCOLA, short for Coherence-Oriented Contrastive Learning for Audio, is a contrastive learning method for musical audio representations designed to capture harmonic and rhythmic coherence between samples at the level of stems composing music tracks. It was introduced as a way to support objective evaluation of generative models for music accompaniment generation, a setting in which established metrics are difficult to interpret. The method couples a stem-level contrastive objective with a bilinear similarity function, and its derived scalar compatibility measure, the COCOLA Score (CCS), is used as a proxy for coherence between conditioning audio and generated accompaniment (Ciranni et al., 2024).

1. Conceptual scope and problem setting

COCOLA is formulated around the observation that musical coherence is not adequately characterized by generic audio similarity alone. Its target quantity is the degree to which two stem mixtures are harmonically and rhythmically compatible. In the original formulation, the method learns an embedding space in which sub-mixes that co-occur within the same short temporal window are treated as coherent, whereas sub-mixes drawn from different windows are treated as incoherent negatives (Ciranni et al., 2024).

The primary use case is accompaniment generation. In that setting, a model must produce stems that remain musically matched to conditioning stems rather than merely realistic in isolation. COCOLA addresses this by operating directly on mixtures of stems and by defining the COCOLA Score as the similarity between embeddings of conditioning and generated audio. The paper presents this as an objective evaluation mechanism for recent accompaniment-generation systems and reports released checkpoints trained on public stem-separated datasets (Ciranni et al., 2024).

A recurrent misconception concerns the front end. The abstract of the original paper states that COCOLA “can input features obtained via Harmonic-Percussive Separation (HPS),” but the detailed description specifies that the published system does not actually use an HPS front end. Instead, audio is converted to a log-mel filterbank spectrogram before being processed by the encoder. This suggests that HPS is part of the conceptual scope of the method, whereas the reported implementation relies on mel-spectrogram inputs (Ciranni et al., 2024).

2. Contrastive formulation and training objective

The training setup begins with a full track $\bar{\mathbf{x}}^k$ separated into $N$ stems $\{\bar{\mathbf{x}}^k_n\}_{n=1}^N$. For each sampled track, a random window $\mathbf{x}^k \in \mathbb{R}^{L \times N}$ of length $L$ is extracted, subject to the constraint that no two windows in the same track overlap by more than ratio $r$. The stems inside that window are then partitioned into two disjoint, nonempty subsets $X^k_1$ and $X^k_2$, which are summed to form two sub-mixes (Ciranni et al., 2024):

$$\mathbf{m}^k_1 = \sum_{n\in X^k_1}\mathbf{x}^k_n,\quad \mathbf{m}^k_2 = \sum_{n\in X^k_2}\mathbf{x}^k_n.$$

Each sub-mix is embedded by a shared encoder $f_\theta$:

$N$0

Similarity is defined bilinearly:

$N$1

The loss is a multi-class cross-entropy, described as InfoNCE-style:

$N$2

Positive pairs are the two sub-mixes drawn from the same window, and negatives are cross-window pairs. By construction, the objective encourages embeddings of stems or sub-mixes that co-occur in time, and therefore share harmonic and rhythmic structure, to be close, while pushing apart sub-mixes from different windows (Ciranni et al., 2024).

3. Encoder, input representation, and optimization protocol

The reported implementation uses 5-second audio chunks at 16 kHz, converted to mel-filterbank spectrograms with 128 mel bins. The backbone is EfficientNet-B0 with dropout $N$3 on all layers. A single linear layer maps the final EfficientNet feature map to a 512-dimensional embedding, and the similarity head is a learnable $N$4 bilinear matrix $N$5 (Ciranni et al., 2024).

Training uses window length $N$6 s and batch size $N$7 windows per batch. The overlap constraint is $N$8. Optimization uses Adam with learning rate $N$9, and additive Gaussian noise with $\{\bar{\mathbf{x}}^k_n\}_{n=1}^N$0 is applied on positive samples. The paper reports that bilinear similarity outperforms cosine similarity and that the model converges in approximately 100 epochs, with the final checkpoint used for evaluation (Ciranni et al., 2024).

The training corpora listed in the detailed summary are MoisesDB, Slakh2100, and CocoChorales, together with a combined “COCOLA All” model trained on all three. The same summary states that MUSDB18-HQ is held out for generalization testing. The dataset descriptions are reported as follows: MoisesDB at approximately 14 hours with an 11-stem taxonomy, Slakh2100 at approximately 145 hours with 34 synthesized stems, CocoChorales at approximately 1,411 hours with 13 synthesized instruments and a tiny subset of 4,000 tracks used, and MUSDB18-HQ at approximately 10 hours with 4 stems (Ciranni et al., 2024).

4. COCOLA Score and accompaniment-generation evaluation

Once the encoder $\{\bar{\mathbf{x}}^k_n\}_{n=1}^N$1 is trained, the COCOLA Score is defined between a conditioning mixture $\{\bar{\mathbf{x}}^k_n\}_{n=1}^N$2 and a generated accompaniment $\{\bar{\mathbf{x}}^k_n\}_{n=1}^N$3 by embedding both signals and applying the same bilinear similarity (Ciranni et al., 2024):

$\{\bar{\mathbf{x}}^k_n\}_{n=1}^N$4

$\{\bar{\mathbf{x}}^k_n\}_{n=1}^N$5

This real-valued score is used as a proxy for how well the harmony and rhythm of the generated accompaniment match the conditioning mixture. In the reported benchmark, CCS is compared against Fréchet Audio Distance (FAD) variants using CLAP, EnCodec, and VGGish features. The comparison involves MSDM, described as the prior multi-source diffusion baseline; CompoNet, described as a new ControlNet-based model; and a Random baseline formed by selecting an unrelated real sub-mix (Ciranni et al., 2024).

The empirical findings reported in the detailed summary are specific. On held-out MUSDB18-HQ, the “COCOLA All” model achieves 90.4% accuracy at detecting which pairs of sub-mixes come from the same window, compared with 50% random, while single-dataset models score below 55%. In the accompaniment-generation benchmark, FAD is reported to favor Random, because Random is real data, and to slightly favor MSDM over CompoNet. By contrast, CCS orders the systems as CompoNet $\{\bar{\mathbf{x}}^k_n\}_{n=1}^N$6 MSDM $\{\bar{\mathbf{x}}^k_n\}_{n=1}^N$7 Random, which the summary states matches perceptual coherence. The same summary reports an upper bound of approximately $\{\bar{\mathbf{x}}^k_n\}_{n=1}^N$8 CCS for ground-truth pairs, approximately $\{\bar{\mathbf{x}}^k_n\}_{n=1}^N$9 for CompoNet on Slakh, and approximately $\mathbf{x}^k \in \mathbb{R}^{L \times N}$0 for MSDM (Ciranni et al., 2024).

The paper further notes that no formal human-listening study was run. The claim is therefore narrower than a listener-study validation: higher CCS is said to correlate with visibly better accompaniment in qualitative checks, and the ranking of models is reported to align with what human listeners would prefer, but no direct statistical evaluation against listener judgments is provided (Ciranni et al., 2024).

5. Use of COCOLA in automatic mashup creation

A later mashup-generation study uses COCOLA as a black-box compatibility estimator between separated stems. In that work, COCOLA is described as “a contrastive model trained to evaluate harmonic and rhythmic coherence between audio segments,” and its output is treated as a scalar compatibility score between two stems. The study does not reproduce the internal loss, feature extraction, or similarity equations of COCOLA; instead, it evaluates whether general-purpose pretrained audio models can approximate the same compatibility relation in a zero-shot way (Delabaere et al., 30 Jul 2025).

The reported pipeline is explicit. A song is first separated into vocals and accompaniment via Demucs. Musical analysis, including tempo, beat/downbeat, key, and segment boundaries, is performed via Allin1. Alignment in tempo and pitch is handled through Pyrubberband. COCOLA is then applied exhaustively over ordered stem pairs. The study uses a subset of FMA containing popular-music tracks only: 21 songs, all in major keys within $\mathbf{x}^k \in \mathbb{R}^{L \times N}$1 semitones of C major—namely B-flat, C, C-sharp/D-flat, and D—with duration constrained to 184–194 s. This yields 42 stems and 420 possible pairings when mixed with 20 other base tracks (Delabaere et al., 30 Jul 2025).

A central result is that compatibility is asymmetric. If $\mathbf{x}^k \in \mathbb{R}^{L \times N}$2 denotes a vocal stem and $\mathbf{x}^k \in \mathbb{R}^{L \times N}$3 an accompaniment stem, then the compatibility score satisfies $\mathbf{x}^k \in \mathbb{R}^{L \times N}$4. The study reports that the role assignment matters: swapping which track serves as vocals and which serves as accompaniment changes the score, sometimes substantially. This indicates that mashup compatibility is not modeled as a symmetric relation on songs but as a role-conditioned relation on stems (Delabaere et al., 30 Jul 2025).

The same work compares COCOLA scores with cosine similarities derived from CLAP and MERT embeddings. The correlations are near zero:

Correlation metric	CLAP embeddings	MERT embeddings
Pearson $\mathbf{x}^k \in \mathbb{R}^{L \times N}$5	0.051	-0.018
Spearman’s rank $\mathbf{x}^k \in \mathbb{R}^{L \times N}$6	0.079	-0.017
Kendall’s $\mathbf{x}^k \in \mathbb{R}^{L \times N}$7	0.053	-0.010

These values are reported as indicating no meaningful correlation between zero-shot embedding similarity and the COCOLA compatibility measure. The authors further state that CLAP cleanly separates vocals from instrumentals, and even synthetic from human voices, which degrades cross-stem compatibility estimation, whereas MERT mixes vocals and instrumentals but still fails to reproduce the harmonic and rhythmic cues captured by COCOLA in the zero-shot regime. A footnote in the mashup paper notes an observed strong correlation between COCOLA scores and subjective mashup quality, but the paper provides no formal listening test, no Mean Opinion Score protocol, and no statistical analysis of human ratings (Delabaere et al., 30 Jul 2025).

6. Limitations, open questions, and disambiguation

Several limitations are explicit in the reported summaries. First, despite the method’s focus on harmonic and rhythmic coherence, the published implementation does not incorporate explicit harmonic-percussive separation features or dedicated rhythm modules; it relies on mel-spectrograms alone. Second, negative pairs drawn from different windows of the same track can still be accidentally coherent if the overlap constraint is not sufficiently strict. Third, the training scale remains limited to public separated datasets totaling less than 2,000 hours, which the summary presents as a motivation for scaling to larger pre-separated or weakly separated corpora. Fourth, no direct human-listening study is provided, leaving the relation between CCS and listener judgments only partially validated (Ciranni et al., 2024).

The original paper also identifies future directions. One is to mine harder negatives more carefully. Another is to use COCOLA as a differentiable “likelihood” or guidance term during diffusion-model inference, directly steering generation toward higher coherence. A plausible implication is that the learned score is intended not only for post hoc evaluation but also as a train-time or inference-time control signal for generative music systems (Ciranni et al., 2024).

The term should also be distinguished from similarly named work in another domain. “CoCo-CoLa: Evaluating and Improving Language Adherence in Multilingual LLMs” introduces a metric for multilingual language adherence, defined there as “Correct Concept - Correct Language,” and concerns output-language selection in multilingual LLMs rather than harmonic and rhythmic coherence in musical audio (Rahmati et al., 18 Feb 2025). The similarity of names does not indicate a shared methodology or application domain.