KuiSCIMA v2.0: Chinese Music Notation Dataset

• The paper presents a machine-readable OMR dataset that integrates three distinct Chinese music notation systems from historical sources.
• It implements a leave-one-edition-out protocol and focal loss training to effectively tackle class imbalance and improve symbol recognition accuracy.
• The dataset supports digital humanities research by providing calibrated, edition-aware benchmarks that facilitate comparative transcription and semi-automatic annotation.

KuiSCIMA Dataset, in its second release as KuiSCIMA v2.0, is a machine-readable optical music recognition dataset derived from Jiang Kui’s Baishidaoren Gequ (白石道人歌曲, 1202), designed for historical Chinese music notations and extended to cover all 109 pieces in the collection. The corpus spans suzipu, lülüpu, and jianzipu, and the accompanying benchmark emphasizes low-resource, class-imbalanced symbol recognition, calibrated probabilistic inference, and robust evaluation across five historical editions. In the reported experiments, the dataset supports improved suzipu and lülüpu baselines, with suzipu Character Error Rate reduced from 10.4% to 7.1% on Shanghai MS and lülüpu Character Error Rate reaching 0.9%, while jianzipu is included at the data level but not yet modeled for OMR (Repolusk et al., 24 Jul 2025).

1. Corpus definition and historical scope

KuiSCIMA v2.0 is defined as the second release of a machine-readable OMR dataset derived from Baishidaoren Gequ, one of the earliest surviving examples of melodized lyrics in Chinese history. The dataset extends the original KuiSCIMA suzipu resource to the entire collection, adds newly annotated lülüpu material, and includes jianzipu instances as part of the corpus. The collection is historically distinctive because it contains three different Chinese notation systems—suzipu, lülüpu, and jianzipu—and survives in multiple later manuscript or xylographic editions rather than in an autograph source (Repolusk et al., 24 Jul 2025).

Five historical editions serve as image sources: Lu, Zhang, Siku Quanshu, Zhu, and Shanghai manuscript. The experimental design uses these editions explicitly, since all reported evaluations follow a leave-one-edition-out scheme. This edition-aware organization is central to the dataset’s structure: the benchmark is not merely a pooled symbol corpus, but a corpus with controlled cross-edition generalization.

The paper also situates the dataset within a more fragile textual and musicological transmission history. All preserved editions are copies of earlier copies; even the earliest extant editions stem from the eighteenth century and can be traced back to the now-missing Tao MS. The paper states that suzipu had already fallen out of practical use long before these copies were made, so the surviving witnesses may contain unreliable or inconsistent renderings. This is important for interpreting KuiSCIMA v2.0: the dataset is a computationally structured representation of historically mediated sources, not a transparent record of an uncontested original notation.

2. Notational coverage and label structure

The dataset’s most distinctive feature is its simultaneous inclusion of three notation systems with substantially different symbol grammars. Suzipu is modeled as a composite notation in which each symbol comprises a pitch component and an optional secondary component. The paper adopts Wu Santu’s system, using 11 pitch classes and 7 secondary classes, where the seven secondary values consist of six proper classes plus absence of a secondary sign. This yields 77 conceptual pitch-by-secondary combinations, of which 66 actually occur in KuiSCIMA. The pitch glyphs are 合, 四, 一, 上, 勾, 工, 凡, 六, 五, 尺, and 高五, while the secondary values are 小住, 丁住, 折, 拽, 大顿, 大住, and None.

Lülüpu is structurally simpler. It is treated as a flat 17-class character recognition problem, with all 17 classes occurring in the dataset. The notation consists of a restricted subset of standard Chinese characters, including pitch names such as 黄钟, 大吕, 太簇, 夹钟, 林钟, 夷则, 南吕, 无射, 应钟, several “qing” variants, the compound 折字, and 高五. Unlike suzipu, lülüpu is not factorized into subcomponents.

Jianzipu is the most complex notation in the corpus. The paper characterizes jianzipu symbols as multi-component composites with very high class complexity. KuiSCIMA v2.0 includes 708 jianzipu instances made up of 98 different annotations, but does not attempt jianzipu OMR in the reported benchmark because the notation is too complex for the available data volume.

A concise summary of the corpus composition is given below.

Notation	Instances / labels	Role in v2.0
Suzipu	7297 non-empty annotated instances; 133 excluded from training; 77 conceptual classes, 66 observed	Main OMR benchmark
Lülüpu	3385 non-empty instances; 17 classes	Added OMR benchmark
Jianzipu	708 instances; 98 annotations	Included in corpus, not modeled

The label distribution is highly uneven for suzipu. The paper explicitly notes severe class imbalance among the 66 observed composite classes, with frequent classes such as Che/None contrasting with rare combinations such as Gong/Dadun or Gong/Xiaozhu. By contrast, lülüpu is described as less problematic from the standpoint of class balance, and class imbalance is not emphasized there. This asymmetry between notation systems explains much of the benchmark design.

3. Data representation, annotation, and split protocol

The operational unit in KuiSCIMA v2.0 is the cropped symbol patch rather than the full page. The paper states that patches are extracted and stored as individual images, accompanied by a JSON file describing their annotations. The JSON contains, at minimum, identifiers, edition metadata, notation type, and the relevant labels: pitch plus secondary for suzipu, a single pitch label for lülüpu, and one of 98 annotation labels for jianzipu (Repolusk et al., 24 Jul 2025).

Annotation is supported by a specialized GUI integrated with methods intended to reduce human effort. The workflow described for suzipu and lülüpu involves detecting and cropping each notation symbol into a patch and assigning the relevant label set. The tool also uses similarity visualization based on UMAP embeddings to surface optically similar instances, supporting annotation consistency. The paper further notes that the original KuiSCIMA work and the “Suzipu Musical Annotation Tool” provide additional procedural detail, while the present work emphasizes the resulting benchmark structure.

For model input, patches are preprocessed to 48×48 pixels. Suzipu training images are resized so that the longest side is uniformly scaled between 30 and 42 pixels, randomly rotated between $-9^\circ$ and $9^\circ$, and cropped to 48×48. Validation and test patches are resized to a longest side of 40 pixels and padded to 48×48. Lülüpu uses a similar regime, with training-side scaling between 33 and 46 pixels and test-time resizing to 40 pixels, but explicitly omits denoising because of the fine structure of Chinese characters.

The benchmark protocol is leave-one-edition-out cross-validation over the five editions. For each held-out test edition, all symbols from the remaining four editions form the development pool, and within that pool 75% of samples of each class are randomly assigned to the training set and 25% to the validation set. For suzipu, this split is performed with respect to the composite class structure relevant to the factorized model; for lülüpu, it is performed over the 17 labels. The scheme is designed to preserve comparable class distributions between training and validation while maintaining genuinely out-of-edition test evaluation.

4. Baseline tasks and benchmark performance

The paper defines the primary use of KuiSCIMA v2.0 as symbol-level classification and clustering for suzipu and lülüpu image patches. Suzipu OMR is factorized into an 11-class pitch classifier and a 7-class secondary classifier, whose outputs are combined into a composite prediction. Lülüpu OMR is formulated as a direct 17-class classification problem. Jianzipu is excluded from modeling in this version because the available corpus does not yet support the notation’s complexity (Repolusk et al., 24 Jul 2025).

All reported classifiers use a compact CNN architecture with three convolutional layers and two fully connected layers, taking 48×48 inputs. Each convolutional layer preserves spatial dimensions through padding and is followed by ReLU, batch normalization, and 2×2 max pooling. The first fully connected layer uses ReLU and 50% dropout, and the output layer has dimension 11, 7, or 17 depending on the target task. The architecture is deliberately small to mitigate overfitting under scarce-data conditions and to permit CPU inference.

Training uses Adam with learning rate $10^{-3}$ for suzipu and $5 \cdot 10^{-4}$ for lülüpu, together with weight decay of $10^{-4}$ on all parameters except biases and batch normalization. If validation loss plateaus for 5 epochs, the learning rate is reduced by a factor of 0.5. The loss is focal loss with $\gamma = 1$, and training instances are sampled uniformly with replacement with respect to the class being learned, a design choice intended to counter pronounced imbalance. Suzipu is trained for 80 epochs with 43 batches of 100 images per epoch; lülüpu is trained for 50 epochs with 21 or 22 batches of 100 images.

On suzipu, the aggregated total CER over all editions and model samples is 6.6% ± 1.2%. For the best model per edition, CER is 5.4% on Lu, 4.6% on Zhang, 5.3% on Siku Quanshu, 9.0% on Zhu, and 7.1% on Shanghai MS. The Shanghai MS result is the headline comparison to the earlier KuiSCIMA baseline, improving from 10.4% to 7.1%.

On lülüpu, aggregated average CER is 1.7% ± 1.9% without artificial data and 0.9% ± 0.9% with artificial data. For the best model per edition, the paper reports 0.3–0.4% on Lu, 0.6% versus 0.3% on Zhang, 0.7% versus 0.4% on Siku Quanshu, 5.0% versus 4.6% on Zhu, and 2.4% versus 0.9% on Shanghai MS, comparing original and artificial-data conditions. The artificial samples are generated from four computer fonts—AR PL UKai CN, AR PL Mingti2L Big5, Noto Sans CJK JP, and Noto Serif CJK JP—yielding 86 additional training images. The paper states that these synthetic additions reduce the variance of the accuracy values and make the models more robust.

The paper also benchmarks against human transcribers and generic OCR. In a suzipu user study, 15 participants naive to both suzipu notation and Chinese achieved an average CER of 15.9% ± 10.0 and a best-case CER of 7.6%, while the OMR models performed significantly better according to a Wilcoxon rank-sum test with $p = 1.9 \cdot 10^{-7} < 0.01$. In timing terms, the suzipu model classified all 1439 Shanghai MS instances in 2.04 ± 0.22 seconds, compared with 9152 ± 2318 seconds for the average human. For lülüpu, Tesseract with chi_tra.traineddata and multiple page segmentation modes yielded CER between 42.1% and 44.0%, far worse than the domain-specific CNN, which also ran much faster.

5. Calibration, edition robustness, and interpretive boundaries

A major addition in KuiSCIMA v2.0 is explicit probability calibration. The paper applies temperature scaling to model logits and reports well-calibrated confidence estimates measured by Expected Calibration Error on 10 equidistant bins, denoted ECE10. For suzipu, ECE10 is below 0.0154; for lülüpu, temperature scaling lowers the maximum ECE10 from 0.0177 to 0.0162; across all classifiers, the reported range is 0.0062 to 0.0162 (Repolusk et al., 24 Jul 2025).

These values indicate that KuiSCIMA v2.0 is not only a recognition benchmark but also a calibrated prediction benchmark. That matters for semi-automatic annotation workflows, where confidence thresholds can be used to route uncertain cases for expert review. The paper explicitly frames the classifiers as “well-calibrated models,” making uncertainty estimation part of the dataset’s benchmark identity rather than an afterthought.

The strongest generalization result is cross-edition rather than cross-notation. Leave-one-edition-out evaluation shows that models trained on four editions transfer effectively to the fifth, though performance consistently drops on the Zhu edition for both suzipu and lülüpu. The paper interprets this as an indication that Zhu’s image distribution differs from the others. This makes edition metadata analytically important: the dataset can support research on domain shift across historical print and manuscript witnesses.

The title’s phrase “cross-notation generalization” requires careful reading. The corpus indeed spans three notation systems, but the reported OMR models are notation-specific, and the paper does not present experiments that train on one notation and test on another or that learn a unified multi-notation representation. This suggests that the cross-notation contribution in v2.0 lies primarily in corpus coverage and in enabling future comparative work, rather than in demonstrating direct transfer across notation systems.

6. Scholarly significance, limitations, and prospective extensions

KuiSCIMA v2.0 occupies a dual position as both a machine learning benchmark and a digital humanities resource. The dataset makes the works of Jiang Kui computationally accessible across all 109 pieces and provides a structured basis for preservation, comparative transcription, and future linkage with large textual repositories such as the Kanseki Repository and the Chinese Text Project, where lyrics may be available without corresponding notation (Repolusk et al., 24 Jul 2025).

At the same time, the paper identifies important limitations. First, the corpus is confined to a single source tradition, Baishidaoren Gequ, and does not yet extend to other suzipu repertories such as Xi’an Guyue or broader qin materials. Second, jianzipu remains outside the current OMR benchmark because its symbol composition is too complex for the available dataset scale. Third, the surviving editions are historically unreliable witnesses copied by scribes likely unfamiliar with suzipu, so the ground truth may necessarily encode editorial decisions rather than uncontested readings. Fourth, class imbalance and data scarcity remain substantial, especially for rare suzipu composite classes and for the Zhu edition’s distinct visual characteristics. Finally, the present release is fundamentally a symbol-level corpus: it does not yet provide full MEI encodings or explicit alignments to lyrics, modes, or higher-level melodic analysis.

The paper outlines several future directions. These include re-encoding KuiSCIMA into a standard notation format such as the Music Encoding Initiative, digitizing other suzipu-based music and related traditions such as Jyutkek I, and developing end-to-end OMR systems for suzipu and lülüpu. The inclusion of labeled jianzipu instances also points toward future work on high-class-count, multi-component recognition. A plausible implication is that KuiSCIMA v2.0 may serve as a bridge from isolated symbol recognition toward structurally richer models of historical Chinese music notation, but the current benchmark remains firmly focused on symbol-level classification under scarce and imbalanced data conditions.