MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement (1905.04874v1)

Abstract: Adversarial loss in a conditional generative adversarial network (GAN) is not designed to directly optimize evaluation metrics of a target task, and thus, may not always guide the generator in a GAN to generate data with improved metric scores. To overcome this issue, we propose a novel MetricGAN approach with an aim to optimize the generator with respect to one or multiple evaluation metrics. Moreover, based on MetricGAN, the metric scores of the generated data can also be arbitrarily specified by users. We tested the proposed MetricGAN on a speech enhancement task, which is particularly suitable to verify the proposed approach because there are multiple metrics measuring different aspects of speech signals. Moreover, these metrics are generally complex and could not be fully optimized by Lp or conventional adversarial losses.

• The paper introduces MetricGAN, redefining the GAN discriminator to output continuous scores that mirror evaluation metrics like PESQ and STOI.
• It demonstrates enhanced training efficiency with faster convergence toward target speech quality and intelligibility scores.
• The approach provides practical benefits for generating speech with customizable metric-driven quality improvements in real-world applications.

Overview of MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement

The paper introduces MetricGAN, a novel approach to address limitations in current Generative Adversarial Networks (GANs) applications for speech enhancement. The primary concern it tackles is the disconnect between adversarial loss functions used in GANs and the evaluation metrics they aim to improve, such as perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI). This disconnect often results in GAN-generated data that does not optimally improve, or even degrades, these evaluation scores.

Contribution of MetricGAN

MetricGAN advances the field by proposing a framework where the GAN discriminator is directly linked with evaluation metrics. Unlike traditional setups where the discriminator's role is merely to differentiate between real and fake data, in MetricGAN, the discriminator is trained to evaluate how closely the generated data corresponds with desired metric scores. This adjustment redefines the discriminator's output from a binary classification to a continuous score that aligns with the target evaluation metrics, effectively transforming it into a learned surrogate of the evaluation metrics.

Implications and Findings

The research demonstrates several key points through experiments on speech enhancement tasks:

• Training Efficiency: MetricGAN significantly enhances training efficiency compared to conventional GAN setups that incorporate a simplistic regression approach with L-p losses. Specifically, it showcases faster convergence (in terms of the number of iterations) towards optimized evaluation scores.
• Score Optimization: By employing discriminators trained to approximate specific evaluation metrics, MetricGAN enables the assignment of arbitrary evaluation scores to the generated data. This feature provides flexibility, allowing MetricGAN to generate speech with desired intelligibility and quality metrics.
• Competitive Performance: When compared with state-of-the-art models, MetricGAN outperforms them across various metrics such as PESQ and STOI, affirming the efficacy of its unique discriminator design.

Theoretical and Practical Implications

From a theoretical perspective, MetricGAN provides a compelling framework for enhancing the generative capacity of GANs by re-purposing the discriminator's role, which could be applied to other domains requiring metric-based optimization. Practically, its application in speech enhancement can significantly improve the quality of speech signals processed in telecommunications and assistive hearing technologies.

The innovative incorporation of evaluation metrics directly into the GAN learning process addresses a critical bottleneck in the current application of GANs for tasks with complex evaluation criteria. Furthermore, by allowing the assignment of specific target metric scores, MetricGAN can potentially pave the way for more nuanced and task-specific generation models in various signal processing applications.

Future Developments

The adaptability of MetricGAN to different metric specifications opens avenues for continued research in multi-metric score optimization and the expansion of GAN-based models to other signal domains. A future trajectory could include refining the model to better handle non-extreme or conflicting metric targets in multi-metric objectives or extending the framework for real-time applications where rapid adaptation to metric changes is crucial.

Conclusively, MetricGAN presents a novel approach to optimize evaluation metrics directly within the GAN framework for speech enhancement, and its findings contribute significantly to the field, offering promising developments for machine learning applications in signal processing.