HarmoF0: Logarithmic Scale Dilated Convolution For Pitch Estimation (2205.01019v2)

Abstract: Sounds, especially music, contain various harmonic components scattered in the frequency dimension. It is difficult for normal convolutional neural networks to observe these overtones. This paper introduces a multiple rates dilated causal convolution (MRDC-Conv) method to capture the harmonic structure in logarithmic scale spectrograms efficiently. The harmonic is helpful for pitch estimation, which is important for many sound processing applications. We propose HarmoF0, a fully convolutional network, to evaluate the MRDC-Conv and other dilated convolutions in pitch estimation. The results show that this model outperforms the DeepF0, yields state-of-the-art performance in three datasets, and simultaneously reduces more than 90% parameters. We also find that it has stronger noise resistance and fewer octave errors. The code and pre-trained model are available at https://github.com/WX-Wei/HarmoF0.

• The paper introduces the MRDC-Conv layer to efficiently capture harmonic patterns in logarithmic scale spectrograms for pitch estimation.
• It demonstrates state-of-the-art accuracy and noise robustness with a significantly reduced parameter count compared to competing methods.
• The approach offers practical benefits for real-time audio processing, transcription, and music information retrieval in resource-constrained environments.

Logarithmic Scale Dilated Convolution For Pitch Estimation

The paper "HarmoF0: Logarithmic Scale Dilated Convolution For Pitch Estimation" introduces a novel approach to fundamental frequency (f0) estimation, which involves exploiting the harmonic structure inherent in audio signals. The authors propose the HarmoF0 framework, a fully convolutional network augmented with a multiple rates dilated causal convolution (MRDC-Conv) technique, to more efficiently capture harmonic patterns in logarithmic scale spectrograms.

Technical Overview

The central technical advancement posited in this research is the MRDC-Conv layer, designed to address the challenges of efficiently identifying harmonic components in audio spectrograms. Traditional convolutional neural networks (CNNs) face difficulties given the sparse distribution of harmonic overtones within the frequency domain. By utilizing dilation rates derived from harmonic series properties, the MRDC-Conv offers a mechanism by which the network can expand its receptive field exponentially, yet still remain congruent with the frequency relationships seen in musical harmonics. This is a direct improvement over fixed rate, dilated convolutions which often fail to respect these integral frequency relationships.

The HarmoF0 architecture includes four convolutional blocks, an MRDC-Conv layer in the first block, and subsequent standard dilated convolution (SD-Conv) layers. The network achieves pitch estimation by converting audible frequencies into a logarithmic scale to maintain constant intervals between harmonics, thus facilitating a dilation rate selection that is harmonically informed.

Computational and Experimental Results

Empirical validation shows that HarmoF0 achieves state-of-the-art performance across multiple datasets, such as MIR-1K, MDB-stem-synth, and PTDB-TUG, with the added benefit of significantly reduced model size compared to other approaches like DeepF0 and CREPE. Specifically, HarmoF0 drastically reduces the number of parameters to 0.377 million, a mere fraction of the CREPE’s 22.2 million parameters, while maintaining or improving pitch estimation accuracy. More interestingly, under challenging conditions, such as high levels of noise, HarmoF0 demonstrated robust performance, producing fewer octave errors relative to competing techniques.

Implications

The implications of HarmoF0 extend into applied audio signal processing domains where pitch estimation is critical, such as automatic transcription, music information retrieval, and real-time audio processing in noisy environments. The reduced parameter count suggests this method is well-suited for deployment in resource-constrained scenarios, while the heightened accuracy and noise robustness indicate potential gains in user-facing music and speech processing applications.

Future Directions

Prospects for further development include extending the model's capabilities beyond monophonic pitch estimation to encompass polyphonic audio analysis, thereby advancing tasks in melody extraction and tracking within complex acoustical environments. With continual improvements in the internal representations captured by models like HarmoF0, the fidelity and applicability of AI-driven sound processing solutions are poised to expand. Additional research might also explore integrating self-supervised learning paradigms to alleviate the dependence on exhaustively labeled datasets, which remains a bottleneck in training sophisticated models for music and speech tasks.

In conclusion, the HarmoF0 framework represents a significant advancement in efficiently utilizing deep learning for audio pitch estimation, combining computational efficiency with robustness, thereby presenting a valuable tool for numerous auditory processing endeavors.