WHAMR!: Noisy and Reverberant Single-Channel Speech Separation (1910.10279v2)

Abstract: While significant advances have been made with respect to the separation of overlapping speech signals, studies have been largely constrained to mixtures of clean, near anechoic speech, not representative of many real-world scenarios. Although the WHAM! dataset introduced noise to the ubiquitous wsj0-2mix dataset, it did not include reverberation, which is generally present in indoor recordings outside of recording studios. The spectral smearing caused by reverberation can result in significant performance degradation for standard deep learning-based speech separation systems, which rely on spectral structure and the sparsity of speech signals to tease apart sources. To address this, we introduce WHAMR!, an augmented version of WHAM! with synthetic reverberated sources, and provide a thorough baseline analysis of current techniques as well as novel cascaded architectures on the newly introduced conditions.

Overview of WHAMR!: Noisy and Reverberant Single-Channel Speech Separation

The paper "WHAMR!: Noisy and Reverberant Single-Channel Speech Separation" introduces an advanced dataset aimed at addressing challenges associated with separating overlapping speech signals in realistic audio environments. Previous research in speech separation has predominantly focused on clean, near-anechoic conditions, which do not adequately reflect real-world scenarios where both noise and reverberation commonly occur. WHAMR! extends the existing WHAM! dataset by incorporating synthetic reverberation, thereby simulating a more authentic auditory landscape for evaluating speech separation algorithms.

The primary contribution of this work lies in the creation of the WHAMR! dataset, which is an augmented version of WHAM! by adding synthetic reverberated sources. The paper outlines the methodology for generating reverberated audio using simulated room impulse responses (RIRs) and noise recorded in diverse urban environments. The introduction of WHAMR! enables a more comprehensive evaluation of speech separation systems in scenarios that include ambient noise and reverberation.

The authors conducted a series of experiments to benchmark the performance of various speech separation systems under clean, noisy, reverberant, and noisy plus reverberant conditions. They employed both bidirectional long short-term memory (BLSTM) networks and temporal convolutional networks (TCN) with learned basis transforms for these tasks. Their findings indicate that reverberation, particularly when combined with noise, poses a significant challenge to separation systems, more so than noise alone.

In the experiments, the BLSTM-based systems showed superior performance compared to TCN systems, which contrasts with some previous literature findings. This unexpected trend suggests that the architecture and training strategies employed play a crucial role in the performance of speech separation models, especially under complex audio conditions. The study further illustrates that leveraging learned feature representations, akin to those used in TasNet, enhances performance across both separation and enhancement tasks.

Moreover, the paper highlights the utility of cascading enhancement and separation modules, which allows for distinct processing of noise and reverberation. The authors demonstrated that segregating these tasks into pre- and post-processing stages, followed by fine-tuning through end-to-end training, improves system performance, particularly in conditions with both noise and reverberation.

The results on WHAMR! form a robust benchmark and offer critical insights into designing more effective speech separation and enhancement systems that can function in acoustically challenging environments. Researchers and practitioners in the field of audio signal processing can leverage the WHAMR! dataset to explore advanced network architectures and optimization techniques that could improve separation quality in real-life acoustic scenarios.

The introduction of WHAMR! represents a meaningful step toward aligning speech separation research with the complexities of natural audio recordings. The empirical findings and methodologies discussed serve as a foundation for further exploration and advancements in adaptive and robust speech processing systems. As speech-based applications continue to expand in everyday environments, datasets like WHAMR! become indispensable for pushing the boundaries of current technology and enhancing the resilience of speech recognition and separation systems across diverse acoustic settings.