---
title: 'LMD-clean: Benchmark for MIDI De-duplication'
url: https://www.emergentmind.com/topics/lmd-clean
type: topic
---

# LMD-clean: Benchmark for MIDI De-duplication

LMD-clean denotes the **Clean MIDI subset of the Lakh MIDI Dataset (LMD)**. In current arXiv usage represented here, it is an explicitly defined benchmark for symbolic-music de-duplication rather than a generic “cleaned” variant in an abstract sense. The subset contains **17,184 MIDI files** and is organized with **artist folders** and **song filenames** so that different versions of the same song appear as repeated filenames with minor variants; using this metadata, the study states that **10,355 of the 17,184 files are duplicates**. LMD-clean therefore serves simultaneously as a benchmark test set, a source of weak ground-truth duplicate groupings, and a controlled proxy for the broader de-duplication problem in **LMD-full**, whose **178,561 files** have unique MD5 hashes but can still contain musically duplicate or near-duplicate content [2509.16662].

## 1. Definition and semantic scope

The defining feature of LMD-clean is not manual musical curation in the sense of complete content verification, but metadata organization that exposes “different versions of the same songs” within the larger LMD collection. The paper illustrates this with repeated filenames such as *Dancing Queen.mid* and *Dancing queen.2.mid*, grouped under the same artist and song identity. This organization makes LMD-clean a practical benchmark for duplicate retrieval, because exhaustive manual annotation over all of LMD-full would be infeasible [2509.16662].

Within the study, two duplicate notions are distinguished. **Hard duplication** is the main target and covers files sharing essentially the same arrangement with only minor differences such as instrument mapping or order, tempo changes, start offset, file length, missing tracks, note-level pitch, duration, or velocity changes, ornamentation differences, different numbers of chord tones in some instrument parts, and transposition when the rest remains nearly identical. **Soft duplication** refers to different arrangements of the same underlying song, where core melody or harmony is preserved but accompaniment style, pitch range, structure, and arrangement differ substantially. This distinction matters because LMD-clean’s artist/title grouping naturally captures “same-song” relationships that can extend beyond strict hard duplicates [2509.16662].

The benchmark is intentionally approximate. The paper explicitly notes that LMD-clean is not perfect ground truth in an absolute sense: some files with identical MIDI contents can have different song labels because of metadata errors or alternate international titles. A plausible implication is that LMD-clean is best understood as a high-utility weak benchmark rather than a final authority on musical identity.

## 2. Construction and benchmark protocol

LMD-clean is used as the held-out evaluation substrate in a broader study of duplicate retrieval for LMD. To prevent exact-file contamination, the authors train their proposed model on **LMD-full excluding all files present in LMD-clean by MD5 match**, calling the result **LMD-filtered**, and split that training corpus **98:1:1**. MIDI is preprocessed with **Octuple encoding** using **MidiTok** [2509.16662].

The evaluation protocol has both **retrieval** and **classification** components. Retrieval quality is measured with **nDCG@all** and **MRR**, asking whether true duplicates are ranked highly. Classification quality is evaluated by thresholding similarity scores and reporting **Precision**, **Recall**, **F1**, and **FN**. For neural methods, the threshold is not chosen by best F1 alone; instead, the paper selects the **lowest threshold such that precision > 0.9**, reflecting the practical asymmetry of de-duplication, where false positives remove potentially valid data [2509.16662].

After pairwise duplicate edges are predicted, the study constructs a graph in which each file is a node and each predicted duplicate relation is an edge. Duplicate groups are then defined as connected components found by **depth-first search**. For filtering, one representative per cluster is retained: specifically, the file with the **highest total note count**. This turns LMD-clean from a pairwise ranking benchmark into a calibration device for dataset-level de-duplication.

## 3. Methods benchmarked on LMD-clean

The study compares three method families on LMD-clean: **rule-based baselines**, **pretrained symbolic retrieval models**, and a proposed contrastive model named **CAugBERT**. The rule-based baselines are **MIDI Encoding Hash**, **Beat Position Entropy**, and **Chroma-DTW**. The pretrained representation models are **MusicBERT_small**, **MusicBERT_base**, **CLaMP-512**, **CLaMP-1024**, **CLaMP2**, and **CLaMP3**. CAugBERT is a **4-layer Transformer** aligned with MusicBERT-small settings, using **sequence length 1024**, **hidden size 512**, and **Octuple encoding** input; for retrieval it uses the **[CLS] token embedding** from the final hidden layer [2509.16662].

CAugBERT is notable because it is trained specifically to mimic duplicate-preserving transformations. Its MIDI variation augmentation includes **Onset Shift `(-2, 2)`**, **Duration Shift `(-4, 4)`**, **Velocity Shift `(-3, 3)`**, **Pitch Octave Shift `(-24, 24)`**, **Inst Order `Shuffle`**, **Inst Mapping `Except Drum`**, **Inst Drop `Less than 50%`**, **Bar Drop `15%`**, **Bar Shift `(1, 4)`**, **Note Drop `15%`**, and **Pitch Transpose `(-6, 6)`**. It also uses neighbor-segment positives, where different **1024-token** segments from the same piece are treated as positive pairs [2509.16662].

The benchmark therefore measures not only exact or near-exact symbolic overlap, but also whether an embedding model can treat musically equivalent or trivially altered files as duplicates. This suggests that LMD-clean is valuable less as a general MIR benchmark than as a targeted stress test for duplicate-aware representation learning.

## 4. Quantitative performance on LMD-clean

The results on LMD-clean separate clearly into ranking strength and high-precision duplicate classification. **CLaMP3** is the best retrieval model, achieving **nDCG@all = 0.697** and **MRR = 0.709**. **CAugBERT** is the best duplicate classifier under the paper’s precision-constrained protocol, with **Precision = 0.903**, **Recall = 0.339**, **F1 = 0.493**, and **FN = 4,623**. The closest competing classifier is **MusicBERT_small**, with **Precision = 0.904**, **Recall = 0.336**, **F1 = 0.490**, and **FN = 4,685** [2509.16662].

| Configuration | Key outcome |
|---|---|
| CLaMP3 | Best ranking: nDCG@all 0.697, MRR 0.709 |
| CAugBERT | Best high-precision classification: Precision 0.903, Recall 0.339, F1 0.493 |
| CLaMP-1024 ∪ CAugBERT | Chosen proposed configuration: Precision 0.899, Recall 0.395, F1 0.548 |

An important ablation compares **BERT without Contrastive** against **CAugBERT**. The former reaches **Recall 0.253**, **F1 0.395**, and **FN 5,630**, whereas CAugBERT improves to **Recall 0.339**, **F1 0.493**, and **FN 4,623**. The paper interprets this as evidence that contrastive learning with augmentations materially improves duplicate detection while maintaining the precision target [2509.16662].

The ensemble results are decisive for the study’s downstream pipeline. The union **CLaMP-1024 ∪ CAugBERT** is selected as the **proposed configuration**, because it has the best two-model F1 and nearly matches the union of all methods. Its reported figures are **Precision 0.899**, **Recall 0.395**, **F1 0.548**, and **FN 3,954**. This makes LMD-clean the calibration benchmark for the final LMD-full filtering strategy.

## 5. From LMD-clean to de-duplication of LMD-full

The paper’s broader claim is that LMD-full is duplicated enough to threaten the validity of random train/validation/test splits. It argues that near-identical versions of the same song can leak across splits, thereby making **validation perplexity / NLL / cross-entropy** artificially optimistic and biasing generation and retrieval benchmarks. LMD-clean provides the only practical way, within the study, to benchmark retrieval methods against grouped same-song versions before applying them to the full corpus [2509.16662].

Using thresholds calibrated on LMD-clean, the study reports three de-duplication options for LMD. The **LMD-clean-query filtering list** queries each piece in LMD-clean against LMD-full and targets popular same-song groups. The **full-LMD proposed configuration** uses **CLaMP-1024 ∪ CAugBERT** and yields **23,566 duplicate clusters** with **68,075 duplicate files**. The **full-LMD conservative configuration**, requiring embedding similarity **\(\ge 0.99\)**, yields **20,797 duplicate clusters** with **38,134 duplicate files** [2509.16662].

| Full-LMD filtering configuration | Reported duplicate files |
|---|---:|
| Proposed configuration | 68,075 |
| Conservative configuration | 38,134 |

The manual validation on LMD-full supports the practical value of the LMD-clean-calibrated thresholds. The authors sample **100 random query MIDI files** having at least one detected duplicate, giving a pool of **506 MIDI files** including queries. At the proposed threshold, **72.9%** are judged duplicates, **6.9%** “similar,” and **20.2%** irrelevant. At the stricter **\(\ge 0.99\)** threshold, both “similar” and irrelevant items drop to **2.22%**. The conservative setting is therefore positioned as the high-confidence option [2509.16662].

## 6. Limitations, failure modes, and terminological ambiguity

LMD-clean is effective precisely because it is imperfect but scalable. Its labels are metadata-derived rather than fully curated musical judgments, and the paper documents several failure modes: identical MIDI contents may have different song names, some metadata can be incorrectly linked to different pieces, and alternate international titles can fragment true duplicate groups. The authors also note that all methods perform below an upper bound partly because neural similarity models handle **hard duplicates** better than **soft duplicates** [2509.16662].

The benchmark’s scope is therefore narrower than the term “clean” may initially suggest. LMD-clean does not certify a deduplicated musical ontology for all of LMD; it provides a tractable benchmark for same-song retrieval and threshold selection. A plausible implication is that it is best used for **duplicate-control infrastructure** rather than as a canonical musical reference set.

There is also terminological ambiguity across arXiv. In the materials considered here, the explicit named resource **“LMD-clean”** appears in symbolic music research on the Lakh MIDI Dataset [2509.16662]. By contrast, **“LMD”** in speaker-verification defense denotes **Learnable Mask Detector**, and the paper does **not** define a variant called “LMD-clean”; the closest reading is training or evaluating LMD on **clean/genuine examples** [2211.00825]. In large-language-model evaluation, **CLEAN-EVAL** constructs cleaner benchmark variants but is not an “LMD-clean” resource [2311.09154]. In medical imaging, large-language-model-assisted cleaning produces refined CT-RATE labels and a planned cleaned release, again without the name “LMD-clean” [2606.22382]. This suggests that, among the works considered here, **LMD-clean most concretely denotes the Clean MIDI subset of LMD**, while other nearby usages are analogical or domain-specific rather than standardized nomenclature.

Source: https://www.emergentmind.com/topics/lmd-clean