Sampling-Frequency-Independent Audio Source Separation Using Convolution Layer Based on Impulse Invariant Method

Abstract: Audio source separation is often used as preprocessing of various applications, and one of its ultimate goals is to construct a single versatile model capable of dealing with the varieties of audio signals. Since sampling frequency, one of the audio signal varieties, is usually application specific, the preceding audio source separation model should be able to deal with audio signals of all sampling frequencies specified in the target applications. However, conventional models based on deep neural networks (DNNs) are trained only at the sampling frequency specified by the training data, and there are no guarantees that they work with unseen sampling frequencies. In this paper, we propose a convolution layer capable of handling arbitrary sampling frequencies by a single DNN. Through music source separation experiments, we show that the introduction of the proposed layer enables a conventional audio source separation model to consistently work with even unseen sampling frequencies.

• The paper's main contribution is the design of a Sampling-Frequency-Independent convolution layer using the impulse invariant method to process audio at arbitrary sampling frequencies.
• It integrates latent analog filters and an aliasing reduction strategy to ensure consistent performance from 8 kHz to 48 kHz in diverse audio applications.
• Experimental results on the MUSDB18-HQ dataset show significant performance improvements and reduced output variance compared to conventional models like Conv-TasNet.

Sampling-Frequency-Independent Audio Source Separation: An Overview

Introduction

The paper "Sampling-Frequency-Independent Audio Source Separation Using Convolution Layer Based on Impulse Invariant Method" (2105.04079) presents a novel approach to audio source separation that addresses a critical limitation of traditional deep neural network (DNN) models. The core challenge is the ability to effectively separate audio sources across various sampling frequencies without relying on distinct models tailored for each specific frequency. Conventional models excel at the sampling frequencies for which they are trained; however, these models lack robustness when applied to unseen frequencies. The authors propose an innovative Sampling-Frequency-Independent (SFI) convolution layer designed to overcome this constraint by leveraging the impulse invariant method, enabling a single DNN to process arbitrary sampling frequencies.

Motivation and Problem Framework

The ability to separate audio sources is foundational to numerous audio processing applications such as music remixing, automatic speech recognition, and music transcription. Given the diversity of these applications, the sampling frequency of audio signals varies significantly, often dictated by the application's requirements. Current state-of-the-art models, including those based on Conv-TasNet, are typically optimized for specific sampling frequencies. The key limitation lies in their lack of adaptability to frequencies outside their training set, posing challenges in real-world scenarios where multiple sampling frequencies coexist.

Proposed Methodology

The paper introduces a Sampling-Frequency-Independent convolution layer that integrates continuous-time impulse responses, allowing the generation of digital filter weights via the impulse invariant method. This approach incorporates latent analog filters within the DNN's architecture, effectively decoupling the model's performance from the constraints of fixed sampling frequencies. The proposed layer is versatile and can be trained with standard backpropagation frameworks such as PyTorch, making it accessible for broad adoption in the field.

Key Concepts:

• Impulse Invariant Method: Utilized to generate digital filters from analog counterparts, preserving frequency characteristics across varying sampling frequencies.
• Latent Analog Filters: Serve as the foundational elements of the SFI convolution layer, providing inherent independence from sampling frequency constraints.
• Aliasing Reduction: Introduced to mitigate aliasing effects by zeroing filter weights above the Nyquist frequency, especially relevant at lower sampling frequencies.

Experimental Evaluation

The authors conducted extensive experiments using the MUSDB18-HQ dataset, evaluating the proposed model's performance in music source separation across different sampling frequencies ranging from 8 kHz to 48 kHz. The model's architecture was validated against established benchmarks such as Conv-TasNet and variations incorporating trainable multi-phase gammatone filters (MP-GTF).

Notable findings include:

• Consistent Performance: The proposed SFI model maintained robust performance across unseen sampling frequencies, significantly outperforming Conv-TasNet as frequency mismatches increased.
• Reduced Variance: Experiments demonstrated lower variance in source separation performance, suggesting enhanced robustness to initialization in comparison to conventional methods.
• Effectiveness of Aliasing Reduction: Critical in maintaining performance at lower sampling frequencies, this mitigation strategy further enhanced the model's applicability in diverse frequency conditions.

Conclusion and Implications

This research advances the field of audio processing by enabling single DNN models to effectively handle a wide range of sampling frequencies, thus broadening their applicability in real-world conditions. Future work could explore additional layers' adaptability within DNN architectures or integration with other audio applications, such as speech separation tasks. The methodology outlined in this paper not only enhances practical efficacy but also opens avenues for developing truly universal audio processing systems.

By providing a scalable solution to the sampling frequency challenge in audio source separation, this work lays the groundwork for more adaptable and resilient machine learning models in audio signal processing domains.