A Fully Convolutional Neural Network for Speech Enhancement (1609.07132v1)

Abstract: In hearing aids, the presence of babble noise degrades hearing intelligibility of human speech greatly. However, removing the babble without creating artifacts in human speech is a challenging task in a low SNR environment. Here, we sought to solve the problem by finding a `mapping' between noisy speech spectra and clean speech spectra via supervised learning. Specifically, we propose using fully Convolutional Neural Networks, which consist of lesser number of parameters than fully connected networks. The proposed network, Redundant Convolutional Encoder Decoder (R-CED), demonstrates that a convolutional network can be 12 times smaller than a recurrent network and yet achieves better performance, which shows its applicability for an embedded system: the hearing aids.

• The paper presents the R-CED architecture that maps noisy speech spectra to clean outputs without relying on traditional noise modeling.
• The approach leverages convolutional layers to expand and compress spectral features, preserving essential speech information while eliminating babble noise.
• Results demonstrate significant gains in intelligibility and efficiency, with the CNN model being approximately 12 times smaller than its RNN counterparts.

A Fully Convolutional Neural Network for Speech Enhancement

The paper "A Fully Convolutional Neural Network for Speech Enhancement" presents a novel approach to tackling the pervasive issue of speech denoising, specifically within environments dominated by babble noise. The authors, Se Rim Park from Carnegie Mellon University and Jin Won Lee from Qualcomm Research, propose an innovative method leveraging convolutional neural networks (CNNs) to improve speech intelligibility under these challenging conditions.

Problem and Approach

Babble noise, often encountered in crowded environments, significantly impacts the intelligibility of speech, creating substantial difficulties for hearing aid users. The conventional methods for noise reduction—typically involving noise model estimation—often fall short in environments with low signal-to-noise ratio (SNR), primarily due to their inadequacies in accurately modeling the complex characteristics of babble noise. This research attempts to circumvent these difficulties by employing supervised learning to map noisy speech spectra directly to clean speech spectra, eliminating the need for noise modeling.

The authors put forth the Redundant Convolutional Encoder Decoder (R-CED) network, a fully convolutional network architecture characterized by its smaller size and higher performance compared to traditional fully connected or recurrent neural networks. This makes the proposed method particularly suitable for implementation in embedded systems like hearing aids, where computational resources are limited.

Methodology

The paper introduces a novel problem formulation where the aim is to devise a neural mapping function leveraging CNNs to convert segments of noisy speech spectra into their clean counterparts, minimizing distortion as measured by the $\ell_2$ norm. Unlike recurrent neural networks (RNN) which address temporal relationships naturally, this paper adapts CNNs to the task by considering temporal segments of noisy spectra for denoising.

The R-CED architecture diverges from the conventional Convolutional Encoder Decoder (CED) networks by expanding the spectral input into a higher-dimensional feature space in the encoder before compressing it back to the target in the decoder, effectively allowing the network to focus on essential features while discarding noise components. This architecture demonstrates superior performance even with fewer parameters, attributed to its ability to preserve critical information through redundant representations.

Results and Evaluation

The empirical evaluation involves a thorough comparison among fully connected neural networks (FNN), RNNs, and CNN architectures across various configurations. Key numerical highlights include the observation that the proposed CNN model achieves comparable or superior denoising performance while being significantly smaller in size—approximately 12 times smaller than RNN counterparts. Such efficiency is crucial for deployment in environments where memory and computational power are constrained.

The paper reports substantial gains in speech intelligibility and quality as quantified by metrics such as Signal to Distortion Ratio (SDR), Short Time Objective Intelligibility (STOI), and Perceptual Evaluation of Speech Distortion (PESQ). The enhancement systems achieve higher scores with the R-CED network, particularly when enhanced with bypass connections, underscoring the importance of architectural innovations that streamline the encoding and decoding processes.

Implications and Future Work

The deployment of the R-CED architecture signifies a progressive step toward real-time speech enhancement under adverse conditions. The findings hold substantial practical implications for the development of more effective and efficient hearing aids. On a theoretical level, this research highlights the potential of CNNs in tasks traditionally dominated by RNNs or other methodologies, encouraging their exploration in auditory and even wider application domains.

Future developments will likely focus on further optimization of the R-CED network, particularly in operation count reduction, which could facilitate even more efficient processing suitable for real-time applications. Exploring adaptive mechanisms or integrating additional contextual modeling might also enhance robustness further, potentially extending applications to diverse auditory environments beyond hearing aids.

Overall, this paper is a significant contribution to the field of speech enhancement and advances the utilization of CNNs in practical, constrained environments.