VoxCeleb2: Deep Speaker Recognition (1806.05622v2)

Abstract: The objective of this paper is speaker recognition under noisy and unconstrained conditions. We make two key contributions. First, we introduce a very large-scale audio-visual speaker recognition dataset collected from open-source media. Using a fully automated pipeline, we curate VoxCeleb2 which contains over a million utterances from over 6,000 speakers. This is several times larger than any publicly available speaker recognition dataset. Second, we develop and compare Convolutional Neural Network (CNN) models and training strategies that can effectively recognise identities from voice under various conditions. The models trained on the VoxCeleb2 dataset surpass the performance of previous works on a benchmark dataset by a significant margin.

• The paper introduces VoxCeleb2, a dataset exceeding 6,000 speakers and 1 million utterances, significantly advancing speaker recognition research.
• The study details CNN-based architectures, notably ResNet-50, which achieve a 3.95% EER on benchmark tests in noisy conditions.
• The findings pave the way for future exploration of deeper networks and enhanced embeddings to boost real-world speaker verification performance.

VoxCeleb2: Deep Speaker Recognition

The paper "VoxCeleb2: Deep Speaker Recognition" presents a comprehensive approach to speaker recognition in unconstrained, noisy environments. Authored by Joon Son Chung, Arsha Nagrani, and Andrew Zisserman from the Visual Geometry Group at the University of Oxford, it introduces key contributions in dataset curation and deep learning models for speaker recognition.

Contributions

The paper's primary contributions are twofold:

1. Introduction of the VoxCeleb2 Dataset: VoxCeleb2 is a large-scale audio-visual speaker recognition dataset compiled using a fully automated pipeline. It comprises over a million utterances from more than 6,000 speakers. This dataset is noted to be several times larger than any publicly available speaker recognition dataset, providing a significant resource for the research community.
2. Development of CNN Models for Speaker Recognition: The authors introduce various CNN architectures and training strategies to recognize speaker identities from voice data under noisy conditions. Models trained on VoxCeleb2 demonstrate superior performance on benchmark datasets compared to previous works.

Dataset and Methodology

VoxCeleb2 Dataset

VoxCeleb2 is curated from open-source media, mainly YouTube, and contains a diverse collection of speakers across 145 nationalities. The dataset includes various real-world noise conditions, such as laughter, cross-talk, and background music. The data collection pipeline involves several stages including candidate selection, downloading videos, face tracking, face and speaker verification, and duplicate removal, among others.

VGGVox System

The VGGVox system is the primary architecture presented for learning speaker embeddings. It involves:

• Trunk Architectures: The researchers experiment with both VGG-M and ResNet-based architectures (ResNet-34 and ResNet-50) for extracting features from spectrogram inputs.
• Training: The networks are trained using a two-stage process where the model is first pre-trained for identification using a softmax loss, followed by fine-tuning with a contrastive loss to learn the embedding.
• Evaluation: The system is evaluated on the VoxCeleb1 dataset with notable improvements in performance metrics such as Equal Error Rate (EER) and $C_{det}$.

Results

The models trained on the VoxCeleb2 dataset exhibit marked improvements in speaker verification performance. Specifically, ResNet-50 based models achieve EERs as low as 3.95% on the original VoxCeleb1 test set, demonstrating the efficacy of deeper networks and the larger training dataset. Furthermore, the paper introduces new evaluation protocols using extended and more comprehensive test sets (VoxCeleb1-E and VoxCeleb1-H), providing a rigorous benchmark for future research.

Implications and Future Work

The introduction of the VoxCeleb2 dataset represents a significant advancement for speaker recognition research, enabling the development of more robust models capable of handling diverse and noisy real-world audio. Practically, this dataset could enhance applications ranging from security systems to customer service bots by improving the reliability of automated speaker recognition systems.

Theoretically, the results suggest that deeper CNN architectures, particularly residual networks, offer substantial gains in embedding learning for speaker recognition. This may motivate further exploration of deeper and more complex network architectures.

Future developments could involve exploring other variations of speaker embeddings, leveraging additional modalities for even more robust performance, and continuously improving dataset diversity and size to cover more real-world scenarios.

Conclusion

The paper "VoxCeleb2: Deep Speaker Recognition" significantly contributes to the field by providing a large-scale, diverse dataset and introducing effective CNN-based models for robust speaker verification. It sets a new standard for dataset size and diversity, and its findings regarding model architectures and training strategies offer valuable insights for future research in speaker recognition.