Chord-Former: Transformer Chord Modeling

• Chord-Former is a family of Transformer-based models for chord-event sequence prediction that capture harmonic transitions, rhythmic patterns, and longer-range dependencies.
• It employs distinct architectural patterns, such as bidirectional Transformer pipelines and encoder-decoder segmentation, to improve automatic chord recognition.
• It leverages structured harmonic representations like MERGE and SPLIT tokens alongside pre-training and specialized decoding to enhance the recognition of complex chord types.

Searching arXiv for papers on “Chord-Former” and closely related chord modeling/recognition work. “Chord-Former” designates a Transformer-centered line of research in harmonic sequence modeling, especially automatic chord recognition (ACR), segment-level chord prediction, and chord-conditioned generation. In the cited literature, the term does not identify a single canonical architecture; rather, it names a family of models that apply attention-based sequence modeling to chord events, chord tokens, or chord-conditioned features. Its intellectual basis is the observation that frame-level temporal smoothing is insufficient for modeling musical dependency structure, whereas sequence models applied at the chord-event level can capture musically meaningful transitions, harmonic rhythm, and longer-range context (Korzeniowski et al., 2018, Kim et al., 27 Apr 2026).

1. Genealogy in chord-sequence modeling

Before Transformer-based formulations became prominent, large-scale chord language modeling had already established two premises that directly motivate Chord-Former-style systems. First, shallow finite-context models are inadequate for realistic harmonic prediction. In a study on 1766 unique songs and 161,796 chord annotations, the best 5-gram model obtained a test average log-probability of 1.87, whereas GRU and LSTM models reached 1.24 and 1.25, respectively; the RNN-based models were also reported to adapt to songs at test time, unlike static N-gram models (Korzeniowski et al., 2018).

Second, temporal modeling becomes musically informative only when moved from audio frames to chord events. A later probabilistic framework explicitly separated the temporal model into a harmonic LLM and a chord duration model, arguing that frame-level temporal models mostly smooth predictions, while chord-level sequence models learn dependencies such as musically meaningful transitions. In that formulation, a GRU-512 LLM achieved -1.29 log-probability, compared with -1.89 for a 4-gram, and combining GRU language and duration models improved weighted chord symbol recall (major/minor) from 0.795 to 0.805 (Korzeniowski et al., 2018).

This lineage suggests that Chord-Former is best understood as the Transformer-era continuation of chord-event modeling: the central architectural shift is not merely from recurrence to self-attention, but from framewise label smoothing to structured harmonic sequence prediction.

2. Canonical architectural patterns

The literature contains several concrete Transformer-based patterns that collectively define the Chord-Former family. Some operate on fixed audio windows and predict framewise chord sequences; others reformulate ACR as autoregressive sequence generation over segments and structured chord tokens.

System	Core formulation	Role in the Chord-Former family
BTC	Bidirectional Transformer on CQT features; sequence-to-sequence prediction over audio windows	Strong frame-level Transformer baseline
Harmony Transformer	Encoder performs segmentation; decoder predicts chord labels for each segment from NNLS chroma	Separates segmentation and recognition
pTE-DS	Transformer encoder-decoder with segment-level autoregressive prediction, SPLIT tokens, and encoder pre-training	Structured segment-level Chord-Former formulation

In the BTC pipeline, raw audio is converted to CQT features and processed by a bidirectional Transformer that uses attention in both forward and backward directions to model the context surrounding each audio frame. Harmony Transformer instead operates on NNLS chroma features and uses transformer blocks in which the encoder performs segmentation and the decoder predicts chord labels for each segment. The most explicit Chord-Former-style formulation in the cited material is the segment-level sequence-to-sequence model that predicts chord sequences auto-regressively rather than frame by frame; it uses a Transformer encoder-decoder, starts from a <SOS> token, and continues until <EOS> or a maximum length is reached (Majchrzak et al., 7 Aug 2025, Kim et al., 27 Apr 2026).

The segment-level model formalizes prediction as

$$\hat{\sigma}_j = \arg\max_{\sigma_j \in \Sigma} P(\sigma_j \mid X_\text{spec}, \sigma_{1:j-1}),$$

contrasting with frame-level classification

$$\hat{y}_i = \arg\max_{y_i \in \mathcal{V}} P(y_i \mid X_\text{spec}).$$

This reformulation is important because it mitigates oversegmentation by detecting chord changes only at segment boundaries (Kim et al., 27 Apr 2026).

3. Harmonic representations and tokenization

A defining issue for Chord-Former systems is how harmony is represented. The cited work describes three major representational strategies: flat chord classes, structured symbolic tokens, and chroma-based harmonic features.

The most developed structured-token formulation is the distinction between MERGE and SPLIT representations. In MERGE, each chord event is represented by a time token and a chord token. In SPLIT, each event is decomposed into a time token, a root token, and a quality token. The token sets are

$$\Sigma_M = \mathbb{T} \cup \mathbb{C} \cup \mathbb{S}$$

and

$$\Sigma_S = \mathbb{T} \cup \mathbb{R} \cup \mathbb{Q} \cup \mathbb{S},$$

with special tokens <SOS>, <EOS>, and <PAD>. SPLIT is specifically motivated by the fact that qualities such as maj6, dim, or 7 can be shared across roots, allowing the model to transfer statistical strength across keys and improving recognition of rare or complex chord types. Inference further uses masking so that valid token-type order is respected (Kim et al., 27 Apr 2026).

A parallel harmonic representation appears in text-to-music generation rather than ACR. MusicGen-Chord replaces melody-based one-hot chroma conditioning with chord-based multi-hot chroma conditioning. For melody conditioning,

$$\mathbf{c}_t \in \{0,1\}^{12} \quad \text{with} \quad \sum_{i=1}^{12} c_{t,i}=1,$$

whereas for chord conditioning,

$$\mathbf{c}_t \in \{0,1\}^{12} \quad \text{with} \quad \sum_{i=1}^{12} c_{t,i} > 1.$$

The paper emphasizes that the pretrained MusicGen weights are directly used and only the conditioning representation is changed (Jung et al., 2024).

A third, symbolic precursor is pitchclass2vec, which encodes a chord from its root and pitch class set through the Cartesian product

$$\mathcal{I}_c = \mathrm{root}_c \times \mathcal{C}_c,$$

with the chord embedding computed as

$$\mathbf{u}_c = \sum_{i \in \mathcal{I}_c} \mathbf{u}_i.$$

That work presents the representation for LSTM-based structure segmentation and states that such compositional embeddings could serve as a foundation for models like Chord-Former. This suggests that Chord-Former research is not tied to a single token ontology; it is a broader program of making harmonic structure explicit in the model input space (Lazzari et al., 2023).

4. Training objectives, decoding, and data regimes

Chord-Former systems inherit two practical constraints from earlier MIR work: harmonic modeling is data-scarce relative to NLP, and exact decoding becomes difficult when the temporal model depends on the full sequence history. The cited literature addresses these constraints through pre-training, structured decoding, data augmentation, and artificial datasets.

The segment-level Transformer model introduces encoder pre-training based on chord similarity. For an anchor segment $X_1$ and comparison segment $X_2$, the encoder is trained so that cosine similarity between embeddings matches chord sequence similarity measured by WCSR with the “mirex” criterion. Its objective is

$$\hat{y}_i = \arg\max_{y_i \in \mathcal{V}} P(y_i \mid X_\text{spec}).$$0

This directly addresses data scarcity and imbalance, especially for non-triads and other infrequent chord types (Kim et al., 27 Apr 2026).

Transformer-based ACR also depends strongly on data protocol. A comparative study of BTC and Harmony Transformer used Artificial Audio Multitracks (AAM), Schubert’s Winterreise Dataset, and the McGill Billboard Dataset under a simplified “MajMin vocabulary” of 12 major, 12 minor, and “no chord” (N). The experiments used six-fold cross-validation, pitch augmentation, and dataset balancing across multi-dataset settings. AAM comprises 3,000 algorithmically composed tracks using real instrument samples and includes precise, perfectly aligned annotations derived from MIDI-aligned generation (Majchrzak et al., 7 Aug 2025).

For history-sensitive temporal models, exact Viterbi decoding can become infeasible. The chord-level GRU recognition framework therefore used Hashed Beam Search, enforcing diversity among beam paths by considering chord sequence rather than frame sequence. Although that system is RNN-based rather than Transformer-based, it provides the immediate decoding precedent for Chord-Former-style event-level inference (Korzeniowski et al., 2018).

5. Empirical behavior

The strongest direct empirical evidence for Chord-Former-style modeling comes from segment-level Transformer ACR. In the reported comparison, the pTE-DS configuration—Transformer encoder-decoder with SPLIT tokens and a pre-trained encoder—achieved WCSR (mirex) 85.7, WCSR (tetrads) 73.2, SQ(over) 92.9, and SQ(mean) 88.6. The BTC baseline obtained 80.8, 65.5, 85.9, and 84.6 on the same measures. The paper emphasizes that the gains are particularly notable for complex and infrequent chord types, and that segmentation quality improves while oversegmentation is reduced (Kim et al., 27 Apr 2026).

The training-data study on Transformer-based ACR adds an important domain-generalization result. On AAM, both BTC and HT perform best when trained on AAM; on Winterreise, performance is lowest and training on AAM alone results in poor recognition; on Billboard, models trained on AAM alone perform comparably to those trained on Winterreise, though not as strongly as Billboard-trained models. The same study concludes that AAM can enrich a smaller training dataset of music composed by a human and can even be used as a standalone training set for a model that predicts chord sequences in pop music if no other data is available (Majchrzak et al., 7 Aug 2025).

Earlier RNN results remain relevant because they isolate the source of the modeling gain that Chord-Former then inherits. The large-scale chord-prediction study found that LSTM and GRU models do not degrade over the course of a song and, after approximately 40 chords, their log-probability actually improves, suggesting adaptation to local context. This provides the pre-Transformer empirical rationale for moving toward architectures capable of modeling longer-range harmonic dependencies (Korzeniowski et al., 2018).

6. Applications, adjacent systems, and misconceptions

Although Chord-Former is primarily a term for Transformer-based chord recognition and harmonic sequence modeling, closely related chord-aware generation systems broaden its scope. MusicGen-Chord conditions an autoregressive Transformer-based music generator on text prompts and chord progressions by replacing one-hot melody chroma with multi-hot chord chroma, and MusicGen-Remixer adds a practical pipeline consisting of BPM and downbeat detection with All-in-One, source separation with Demucs, chord extraction with BTC, rhythmic alignment through dynamic time-warping, and final mixing. Both are integrated into Replicate’s web-UI via cog (Jung et al., 2024).

Transformer-based conditioning is also used in symbolic generation. A study of five Transformer-based strategies for chord-conditioned melody and bass generation found that chord-conditioning improves the replication of stylistic pitch content and chord tone usage characteristics, particularly for the bass-first model (Salem et al., 11 Nov 2025). By contrast, a decomposed Retrieval-Edit-Rerank framework for chord generation is not Transformer-based; it separates stylistic retrieval, music-theoretic feasibility, and soft preference handling into explicit stages. Its role in the Chord-Former landscape is therefore contrastive: it demonstrates that some chord-generation problems may benefit from modular constraint enforcement rather than monolithic sequence models (He et al., 8 May 2026).

A recurring misconception is terminological rather than musical. “Chord” also names a distributed hash table protocol used in peer-to-peer systems, with a separate literature on successor lists, stable bases, and Alloy-based correctness proofs. That usage is unrelated to the MIR meaning of Chord-Former and should be kept distinct (Zave, 2015).

Taken together, the cited work indicates that Chord-Former is less a single architecture than a research direction defined by four commitments: modeling at the chord-event or segment level rather than only at the frame level; using structured harmonic representations; leveraging Transformer sequence modeling for longer-range context; and evaluating both recognition accuracy and harmonic plausibility under increasingly strict criteria. A plausible implication is that future Chord-Former systems will extend the current segment-level, structured-token, and pre-training schemes to richer chord vocabularies, finer temporal alignment, and more explicit integration of harmonic function, duration, and generation control.