Melody Accuracy in Music Research

• Melody Accuracy is a quantitative measure that evaluates how well an estimated melody aligns with a target sequence using metrics like Raw Pitch Accuracy, Raw Chroma Accuracy, and Overall Accuracy.
• Recent advances employ deep learning architectures such as CRNNs, Bayesian regression models, and multi-task learning to enhance robustness under polyphonic and noisy conditions.
• Beyond frame-level metrics, evaluation now incorporates note-level transcription, perceptual assessments, and hybrid similarity measures to better capture musical expressiveness and structure.

Melody accuracy refers to the quantitative assessment of how faithfully an estimated or generated melody matches a ground-truth or target melodic sequence, typically in terms of pitch, timing, note values, voicing, and other musically salient attributes. In music information retrieval (MIR), melody accuracy is used to benchmark the performance of algorithms in melody extraction, transcription, conversion, and related tasks across monophonic and polyphonic musical audio. Accurate melody estimation is critical for automatic transcription, music generation, singing voice conversion, and downstream MIR applications.

1. Mathematical Definitions and Standard Metrics

Frame-level accuracy metrics dominate the evaluation of melody extraction. The most widely used are Raw Pitch Accuracy (RPA), Raw Chroma Accuracy (RCA), Overall Accuracy (OA), Voice Recall (VR), and Voice False Alarm (VFA), as formalized in mir_eval and in “Student-t Networks for Melody Estimation” (Gupta et al., 2021), “Regression-based Melody Estimation with Uncertainty Quantification” (Saxena et al., 8 May 2025), and “Toward Expressive Singing Voice Correction” (Luo et al., 2020). Assume $N$ total frames, $$v_{\mathrm{gt}}(n), v_{\mathrm{est}}(n) \in \{0,1\}$$ are ground-truth/estimated voicing indicators, $f_{\mathrm{gt}}(n), f_{\mathrm{est}}(n)$ are ground-truth/estimated F₀, and $\delta=50$ cents is the default tolerance.

• Raw Pitch Accuracy (RPA):

$$\mathrm{RPA} = \frac{\sum_{n=1}^{N} v_{\mathrm{gt}}(n) \mathbf{1}\left( | \mathrm{cents}(f_{\mathrm{est}}(n)) - \mathrm{cents}(f_{\mathrm{gt}}(n)) | < \delta \right ) }{ \sum_{n=1}^{N} v_{\mathrm{gt}}(n) }$$

Measures the proportion of voiced reference frames where the estimated pitch is close (within $\delta$).

• Raw Chroma Accuracy (RCA):

$$\mathrm{RCA} = \frac{\sum_{n=1}^{N} v_{\mathrm{gt}}(n)\, \mathbf{1} \left( \mathrm{chroma}\left( f_{\mathrm{est}}(n)\right) = \mathrm{chroma}\left(f_{\mathrm{gt}}(n)\right) \right) }{ \sum_{n=1}^{N} v_{\mathrm{gt}}(n) }$$

Measures pitch-class (modulo-octave) match on voiced frames.

• Overall Accuracy (OA):

Incorporates correct pitch for voiced frames and correct voicing for unvoiced frames (Luo et al., 2020):

$$\mathrm{OA} = \frac{ \#(\text{correct voiced-pitch} \cup \text{correct unvoiced}) }{ N }$$

• Voice Recall (VR) and Voice False Alarm (VFA):

$$\mathrm{VR} = \frac{ \#(\text{ref voiced} \cap \text{est voiced}) }{ \#(\text{ref voiced}) }$$

$$\mathrm{VFA} = \frac{ \#(\text{ref unvoiced} \cap \text{est voiced}) }{ \#(\text{ref unvoiced}) }$$

For note-level transcription, evaluation typically uses onset, offset, and pitch F1 scores with strict temporal and pitch error tolerances (e.g., $$v_{\mathrm{gt}}(n), v_{\mathrm{est}}(n) \in \{0,1\}$$0ms, $$v_{\mathrm{gt}}(n), v_{\mathrm{est}}(n) \in \{0,1\}$$1 semitones) (Kim et al., 18 Feb 2025).

2. Algorithmic Advances and Model Architectures

Progress in melody accuracy is tightly linked to neural network design and tailored input representations. Notable developments include:

• CRNN and Teacher-Student Architectures: The CRNN-based "Student-t Networks" (Gupta et al., 2021) employ convolutional ResNet blocks and bi-directional LSTMs with semi-supervised pseudo-labeling. This approach yields clear improvements in frame-level RPA/RCA on polyphonic benchmarks.
• Regression-Based Methods with Uncertainty Quantification: Reformulating melody estimation as a regression problem with histogram-based losses and Bayesian treatment for voiced/unvoiced classification improves both accuracy and trustworthiness of outputs. The Bayesian method (M3) achieves RPA up to 96.1% (MIREX05) and OA up to 99.5%, with well-calibrated uncertainty estimates (Saxena et al., 8 May 2025).
• Input Representations Exploiting Harmonic Structure: TONet introduces the Tone-CFP input (frequency bin permutation that groups pitch-class harmonics) and a tone–octave factorization with fusion network, yielding consistent improvements in RCA (tone errors) and ROA (octave errors) compared to prior backbones (Chen et al., 2022).
• Joint Estimation Networks: JEPOO employs multi-task learning on pitch, onset, and offset, with Pareto-modulated loss and weight regularization, robustly outperforming single-task or naive joint models across single-pitch and multi-pitch datasets (Wei et al., 2023).

3. Melody Accuracy under Polyphonic and Noisy Conditions

Polymelodic audio and background accompaniment pose significant challenges. Recent advances exploit self-supervised pretrained backbones and robust fine-tuning:

• SSL-Based Feature Extractors: WavLM and HuBERT backbones, fine-tuned on singing plus randomly mixed background music (BGM), combined via layer-wise learnable weighting and refined by feed-forward Transformer blocks, substantially improve melody accuracy in noisy SVC settings—including at 0dB SNR. Metrics: F₀ RMSE down to 0.176 and F₀ correlation 0.950 (Chen et al., 7 Feb 2025).
• Adversarial and Multi-Discriminator Training: SVC frameworks calculate reconstruction and adversarial losses over spectrograms and embeddings, further enhancing melody and content preservation under challenging conditions (Chen et al., 7 Feb 2025).
• Empirical Benchmarks: Across polyphonic datasets (ADC2004, MIREX05), regression and multi-branch methods consistently yield absolute improvements of 5–10 points in RPA/OA over baseline classification networks (Saxena et al., 8 May 2025, Chen et al., 2022).

4. Beyond Frame-Level Metrics: Note-Level and Expressive Accuracy

Melody accuracy is also measured at the note level and increasingly incorporates rhythmic, durational, and expressive aspects:

• Note-Level F1 and Value Metrics: Time-aligned score generation frameworks define complex correctness sets (onset+offset+pitch+value) and introduce note-value accuracy and mean-squared error (MSE_NV), incentivizing algorithms to capture not just pitch, but also temporal and durational structure (Kim et al., 18 Feb 2025). Symbolic error rates (Levenshtein-based) assess joint pitch–value alignment.
• Subjective/Perceptual Correlates: Studies demonstrate that high frame-wise pitch accuracy (RPA, RCA, OA) does not necessarily align with perceptual quality in synthesis or correction contexts. Voicing coverage (VR, VFA) and smooth, continuous pitch contours have higher perceptual salience (Luo et al., 2020).
• Hybrid and Weighted Metrics: Calls have been made to design hybrid metrics that combine pitch accuracy, voicing coverage, and contour continuity (e.g., mean length of voiced segments, smoothness), or use graded (rather than binary) tolerance penalties (Luo et al., 2020).

5. Alternate Similarity and Accuracy Measures

Alternative, non-framewise assessment approaches include geometric and symbolic matching:

• Geometric Similarity Measures: The t-monotone matching cost, defined on time-pitch sequences in ℝ², provides a many-to-many L₁-matching between reference and query melodies. Associated scaling and compression problems are solved via combinatorial optimization and produce a normalized "melodic accuracy" score (1–D/D_max) suitable for quantifying score–performance or query–reference alignment (Caraballo et al., 2022).
• Contrastive Retrieval Accuracy: In generation models that lack explicit melody-following metrics (e.g., MG²), retrieval recalls (R@1, R@5, mAP@10) over melody embeddings in a shared text–audio–melody space serve as an indirect validation that melodies and waveforms encode consistent semantics, but do not directly yield a "melody accuracy" versus ground-truth (Wei et al., 2024).

6. Limitations, Perception, and Ongoing Challenges

Standard melody accuracy metrics pose several limitations:

• Insensitivity to Contour Continuity and Expressivity: Frame-level, binarized pitch metrics ignore smoothness, contour breaks, and note-level structure—deficiencies highlighted by low correlation with human judgments (Luo et al., 2020).
• Discrepancies Between Objective and Subjective Quality: Systems that maximize RPA/RCA at the expense of continuity or voicing often score poorly in fluency and naturalness in listening tests; over-voicing and continuous estimators often provide superior perceptual outcomes.
• Lack of Expressivity in Metric Design: Metrics typically do not account for performance nuances, ambiguous boundary regions, or expressive pitch modulations.

To address these, recommendations include developing explicit ambiguous-region annotations, designing weighted or continuous tolerance metrics, incorporating continuity and smoothness measures, and distinguishing between classifier-type and continuous estimators depending on application (Luo et al., 2020).

7. Summary of Empirical Melody Accuracy Results

The following table (abridged, see cited works for extended datasets/columns) aggregates reported performance for prominent models and datasets using principal evaluation metrics:

Method	Dataset	RPA (%)	RCA (%)	OA (%)	Note-F1*	F₀ RMSE	F₀ CORR
Student-t (semi-sup)	ADC2004	33.7	41.9	–	–	–	–
TONet (full)	ADC2004	82.6	82.9	82.6	–	–	–
Regression M3 (Bayes)	MIREX05	96.1	–	99.5	–	–	–
SVC-SSL (WavLM)	BGM 0dB	–	–	–	–	0.199	0.935
JEPOO (SP/MP)	MAPS/MAESTRO	–	–	–	81.6–93.0	–	–
T3MS (note-level)	HSD	–	–	–	(Full) 0.514	–	–

*Note-F1 refers to onset+offset+pitch match at standard tolerances.

Absolute accuracy values are contingent on dataset difficulty, evaluation protocol, and the voicing/notation structure.

---

In conclusion, melody accuracy is a multifaceted construct with both algorithmic and perceptual dimensions. While frame-wise metrics (RPA, RCA, OA) remain dominant, the field is evolving to embrace regression-based, note-level, geometric, and perceptually aligned evaluation strategies. New models increasingly exploit robust feature extraction, hierarchical representations, multi-task learning, and uncertainty quantification to optimize and robustly interpret melody accuracy across diverse MIR tasks and audio conditions.