LMD-clean: Benchmark for MIDI De-duplication

• LMD-clean is the clean MIDI subset of the Lakh MIDI Dataset, organized via metadata to expose different versions of the same song for duplicate retrieval.
• It distinguishes hard duplicates, with minor modifications, from soft duplicates, which are varied arrangements of the same underlying song.
• The benchmark supports a de-duplication pipeline using retrieval and classification metrics, with models like CAugBERT and CLaMP achieving high precision.

LMD-clean denotes the Clean MIDI subset of the Lakh MIDI Dataset (LMD). In current arXiv usage represented here, it is an explicitly defined benchmark for symbolic-music de-duplication rather than a generic “cleaned” variant in an abstract sense. The subset contains 17,184 MIDI files and is organized with artist folders and song filenames so that different versions of the same song appear as repeated filenames with minor variants; using this metadata, the study states that 10,355 of the 17,184 files are duplicates. LMD-clean therefore serves simultaneously as a benchmark test set, a source of weak ground-truth duplicate groupings, and a controlled proxy for the broader de-duplication problem in LMD-full, whose 178,561 files have unique MD5 hashes but can still contain musically duplicate or near-duplicate content (Choi et al., 20 Sep 2025).

1. Definition and semantic scope

The defining feature of LMD-clean is not manual musical curation in the sense of complete content verification, but metadata organization that exposes “different versions of the same songs” within the larger LMD collection. The paper illustrates this with repeated filenames such as Dancing Queen.mid and Dancing queen.2.mid, grouped under the same artist and song identity. This organization makes LMD-clean a practical benchmark for duplicate retrieval, because exhaustive manual annotation over all of LMD-full would be infeasible (Choi et al., 20 Sep 2025).

Within the study, two duplicate notions are distinguished. Hard duplication is the main target and covers files sharing essentially the same arrangement with only minor differences such as instrument mapping or order, tempo changes, start offset, file length, missing tracks, note-level pitch, duration, or velocity changes, ornamentation differences, different numbers of chord tones in some instrument parts, and transposition when the rest remains nearly identical. Soft duplication refers to different arrangements of the same underlying song, where core melody or harmony is preserved but accompaniment style, pitch range, structure, and arrangement differ substantially. This distinction matters because LMD-clean’s artist/title grouping naturally captures “same-song” relationships that can extend beyond strict hard duplicates (Choi et al., 20 Sep 2025).

The benchmark is intentionally approximate. The paper explicitly notes that LMD-clean is not perfect ground truth in an absolute sense: some files with identical MIDI contents can have different song labels because of metadata errors or alternate international titles. A plausible implication is that LMD-clean is best understood as a high-utility weak benchmark rather than a final authority on musical identity.

2. Construction and benchmark protocol

LMD-clean is used as the held-out evaluation substrate in a broader study of duplicate retrieval for LMD. To prevent exact-file contamination, the authors train their proposed model on LMD-full excluding all files present in LMD-clean by MD5 match, calling the result LMD-filtered, and split that training corpus 98:1:1. MIDI is preprocessed with Octuple encoding using MidiTok (Choi et al., 20 Sep 2025).

The evaluation protocol has both retrieval and classification components. Retrieval quality is measured with nDCG@all and MRR, asking whether true duplicates are ranked highly. Classification quality is evaluated by thresholding similarity scores and reporting Precision, Recall, F1, and FN. For neural methods, the threshold is not chosen by best F1 alone; instead, the paper selects the lowest threshold such that precision > 0.9, reflecting the practical asymmetry of de-duplication, where false positives remove potentially valid data (Choi et al., 20 Sep 2025).

After pairwise duplicate edges are predicted, the study constructs a graph in which each file is a node and each predicted duplicate relation is an edge. Duplicate groups are then defined as connected components found by depth-first search. For filtering, one representative per cluster is retained: specifically, the file with the highest total note count. This turns LMD-clean from a pairwise ranking benchmark into a calibration device for dataset-level de-duplication.

3. Methods benchmarked on LMD-clean

The study compares three method families on LMD-clean: rule-based baselines, pretrained symbolic retrieval models, and a proposed contrastive model named CAugBERT. The rule-based baselines are MIDI Encoding Hash, Beat Position Entropy, and Chroma-DTW. The pretrained representation models are MusicBERT_small, MusicBERT_base, CLaMP-512, CLaMP-1024, CLaMP2, and CLaMP3. CAugBERT is a 4-layer Transformer aligned with MusicBERT-small settings, using sequence length 1024, hidden size 512, and Octuple encoding input; for retrieval it uses the [CLS] token embedding from the final hidden layer (Choi et al., 20 Sep 2025).

CAugBERT is notable because it is trained specifically to mimic duplicate-preserving transformations. Its MIDI variation augmentation includes Onset Shift (-2, 2), Duration Shift (-4, 4), Velocity Shift (-3, 3), Pitch Octave Shift (-24, 24), Inst Order Shuffle, Inst Mapping Except Drum, Inst Drop Less than 50%, Bar Drop 15%, Bar Shift (1, 4), Note Drop 15%, and Pitch Transpose (-6, 6). It also uses neighbor-segment positives, where different 1024-token segments from the same piece are treated as positive pairs (Choi et al., 20 Sep 2025).

The benchmark therefore measures not only exact or near-exact symbolic overlap, but also whether an embedding model can treat musically equivalent or trivially altered files as duplicates. This suggests that LMD-clean is valuable less as a general MIR benchmark than as a targeted stress test for duplicate-aware representation learning.

4. Quantitative performance on LMD-clean

The results on LMD-clean separate clearly into ranking strength and high-precision duplicate classification. CLaMP3 is the best retrieval model, achieving nDCG@all = 0.697 and MRR = 0.709. CAugBERT is the best duplicate classifier under the paper’s precision-constrained protocol, with Precision = 0.903, Recall = 0.339, F1 = 0.493, and FN = 4,623. The closest competing classifier is MusicBERT_small, with Precision = 0.904, Recall = 0.336, F1 = 0.490, and FN = 4,685 (Choi et al., 20 Sep 2025).

Configuration	Key outcome
CLaMP3	Best ranking: nDCG@all 0.697, MRR 0.709
CAugBERT	Best high-precision classification: Precision 0.903, Recall 0.339, F1 0.493
CLaMP-1024 ∪ CAugBERT	Chosen proposed configuration: Precision 0.899, Recall 0.395, F1 0.548

An important ablation compares BERT without Contrastive against CAugBERT. The former reaches Recall 0.253, F1 0.395, and FN 5,630, whereas CAugBERT improves to Recall 0.339, F1 0.493, and FN 4,623. The paper interprets this as evidence that contrastive learning with augmentations materially improves duplicate detection while maintaining the precision target (Choi et al., 20 Sep 2025).

The ensemble results are decisive for the study’s downstream pipeline. The union CLaMP-1024 ∪ CAugBERT is selected as the proposed configuration, because it has the best two-model F1 and nearly matches the union of all methods. Its reported figures are Precision 0.899, Recall 0.395, F1 0.548, and FN 3,954. This makes LMD-clean the calibration benchmark for the final LMD-full filtering strategy.

5. From LMD-clean to de-duplication of LMD-full

The paper’s broader claim is that LMD-full is duplicated enough to threaten the validity of random train/validation/test splits. It argues that near-identical versions of the same song can leak across splits, thereby making validation perplexity / NLL / cross-entropy artificially optimistic and biasing generation and retrieval benchmarks. LMD-clean provides the only practical way, within the study, to benchmark retrieval methods against grouped same-song versions before applying them to the full corpus (Choi et al., 20 Sep 2025).

Using thresholds calibrated on LMD-clean, the study reports three de-duplication options for LMD. The LMD-clean-query filtering list queries each piece in LMD-clean against LMD-full and targets popular same-song groups. The full-LMD proposed configuration uses CLaMP-1024 ∪ CAugBERT and yields 23,566 duplicate clusters with 68,075 duplicate files. The full-LMD conservative configuration, requiring embedding similarity $\ge 0.99$, yields 20,797 duplicate clusters with 38,134 duplicate files (Choi et al., 20 Sep 2025).

Full-LMD filtering configuration	Reported duplicate files
Proposed configuration	68,075
Conservative configuration	38,134

The manual validation on LMD-full supports the practical value of the LMD-clean-calibrated thresholds. The authors sample 100 random query MIDI files having at least one detected duplicate, giving a pool of 506 MIDI files including queries. At the proposed threshold, 72.9% are judged duplicates, 6.9% “similar,” and 20.2% irrelevant. At the stricter $\ge 0.99$ threshold, both “similar” and irrelevant items drop to 2.22%. The conservative setting is therefore positioned as the high-confidence option (Choi et al., 20 Sep 2025).

6. Limitations, failure modes, and terminological ambiguity

LMD-clean is effective precisely because it is imperfect but scalable. Its labels are metadata-derived rather than fully curated musical judgments, and the paper documents several failure modes: identical MIDI contents may have different song names, some metadata can be incorrectly linked to different pieces, and alternate international titles can fragment true duplicate groups. The authors also note that all methods perform below an upper bound partly because neural similarity models handle hard duplicates better than soft duplicates (Choi et al., 20 Sep 2025).

The benchmark’s scope is therefore narrower than the term “clean” may initially suggest. LMD-clean does not certify a deduplicated musical ontology for all of LMD; it provides a tractable benchmark for same-song retrieval and threshold selection. A plausible implication is that it is best used for duplicate-control infrastructure rather than as a canonical musical reference set.

There is also terminological ambiguity across arXiv. In the materials considered here, the explicit named resource “LMD-clean” appears in symbolic music research on the Lakh MIDI Dataset (Choi et al., 20 Sep 2025). By contrast, “LMD” in speaker-verification defense denotes Learnable Mask Detector, and the paper does not define a variant called “LMD-clean”; the closest reading is training or evaluating LMD on clean/genuine examples (Chen et al., 2022). In large-language-model evaluation, CLEAN-EVAL constructs cleaner benchmark variants but is not an “LMD-clean” resource (Zhu et al., 2023). In medical imaging, large-language-model-assisted cleaning produces refined CT-RATE labels and a planned cleaned release, again without the name “LMD-clean” (Yamagishi et al., 21 Jun 2026). This suggests that, among the works considered here, LMD-clean most concretely denotes the Clean MIDI subset of LMD, while other nearby usages are analogical or domain-specific rather than standardized nomenclature.