---
title: 'VoxBlink2: Large-Scale Audio-Visual Corpus'
url: https://www.emergentmind.com/topics/voxblink2
type: topic
---

# VoxBlink2: Large-Scale Audio-Visual Corpus

Search arXiv for VoxBlink2, VoxBlink, and NIST SRE24 usages to ground the article in recent papers.
VoxBlink2 is a large-scale audio-visual speaker recognition corpus introduced as the successor to VoxBlink and accompanied by an Open-Set Speaker Identification (OSSI) benchmark. It was designed to push speaker recognition toward data scales more typical of face recognition, and to support both conventional speaker verification and open-set identification with thousands of enrolled and unknown speakers. The dataset contains **111,284 speakers**, **2,097,062 videos**, **9,904,382 utterances**, and **16,672 hours**, all sourced from **YouTube** with paired **audio and video** [2407.11510].

## 1. Dataset definition and scale

VoxBlink2 is explicitly presented as a **large audio-visual speaker recognition dataset** built from “in-the-wild” YouTube material. Each example is an **utterance** paired with the corresponding **video clip**, and the collection targets users who use **their own photos as avatars**. Relative to earlier public corpora, the dataset is intended to increase speaker count, temporal spread, and scenario diversity rather than merely add more utterances [2407.11510].

Its scale is the main defining property. VoxBlink2 expands the original VoxBlink from **38,065 speakers**, **372,084 videos**, **1,455,190 utterances**, and **2,135 hours** to **111,284 speakers**, **2,097,062 videos**, **9,904,382 utterances**, and **16,672 hours**. It also exceeds VoxCeleb2 in speaker count, videos, utterances, and total hours [2407.11510].

| Dataset | Speakers | Utterances |
|---|---:|---:|
| VoxCeleb2 | 5,994 | 1,092,009 |
| VoxBlink | 38,065 | 1,455,190 |
| VoxBlink2 | 111,284 | 9,904,382 |

The paper also reports several summary statistics that are relevant for corpus characterization. VoxBlink2 has an **average of 89 utterances per speaker**, an **average utterance duration of 6.0 s**, and an **average time span per speaker of 786 days**. The predecessor VoxBlink had an average span of **440 days**. This suggests that VoxBlink2 was designed not only for scale but also for broader longitudinal variability [2407.11510].

A recurrent misconception is that VoxBlink2 should be interpreted primarily as a blink-specific dataset because of its name. The published description instead defines it as a **speaker recognition corpus** and benchmark. Downstream speaker-recognition papers treat it as a large-scale supervision source for audio and visual identity learning rather than as a corpus built around explicit blink dynamics [2407.11510].

## 2. Collection pipeline and annotation refinement

The collection pipeline is multi-stage and explicitly audio-visual. The initial candidate retrieval uses a **long keyword list spanning 18 languages** and collects **over 6 million 1-minute videos** from YouTube users with photo avatars. This broad multilingual retrieval is one of the main differences from the original VoxBlink, which relied more heavily on short segments [2407.11510].

Frame extraction is performed at **25 fps**. Face detection uses **MobileNet**, and single-person video tracks are built using a minimum **Intersection over Union (IOU)** threshold between consecutive units so that each facial track contains only one person. Face identity assignment then uses a **pre-trained ArcFace classifier**. The authors explicitly state that moving from a verification-style stage to a stronger **identification** stage improved data purity [2407.11510].

The audio side of curation is not treated independently. VoxBlink2 applies an **audio-visual speaker diarization model** together with an **overlap speech detection model** to partition active speech segments and remove silent and overlapped speech segments. This is a substantive design choice: the corpus is not merely face-filtered video, but a collection of active speaking segments with overlap suppression [2407.11510].

A second branch further improves identity purity by training a face classifier on mined data. That branch uses **ResNet-IRSE50**, compares face embeddings against the user’s avatar photo, samples up to **10 faces per speaker** using cosine-similarity-based weighted sampling, and appends **ArcFace** to the encoder for classifier training on about **200K candidate face data**. The reported outcome is a manual accuracy increase from **72%** for VoxBlink to **92%** for VoxBlink2, based on manual assessment of **50 randomly sampled speakers** [2407.11510].

This quality estimate should be interpreted carefully. The improvement from **72%** to **92%** is explicit, but the audit is based on a small manual sample. A plausible implication is that the dataset is materially cleaner than VoxBlink, but the paper does not present a large-scale label audit or a demographic error analysis [2407.11510].

## 3. Training regimes and speaker verification performance

VoxBlink2 is used primarily as a **training** corpus rather than as a standalone benchmark for conventional train/dev/test speaker verification splits. In the main speaker verification experiments, the training datasets are **VoxBlink2 (VB2)** and **VoxCeleb2 (VC2)**, with evaluation on **VoxCeleb1** test sets. Acoustic features are **80-dimensional log Mel-filterbank energies** with **25 ms** frame length, **10 ms** hop size, and fixed input lengths of **200 or 500 frames**. The speaker encoders include **ResNet34**, **ResNet50**, **ResNet100**, **ResNet221**, and **ResNet293**, with **ASP**, **TSP**, and **SimAM** variants [2407.11510].

The paper compares two training strategies. The first is a **Mix-FT** strategy: pre-train on VC2 and fine-tune on mixed data. The second is **pre-train on VB2 and fine-tune on VC2**. The latter is the stronger result. Using **ResNet100-ASP + SimAM**, the VC2 baseline gives **0.606 EER** and **0.052 minDCF\_{0.01}** on VoxCeleb1-O, whereas **VB2 pre-train + VC2 fine-tune** gives **0.340 EER** and **0.026 minDCF\_{0.01}**, which the paper describes as a **43.4% EER reduction** relative to the VC2 baseline [2407.11510].

Scaling analysis is central to the paper’s interpretation of VoxBlink2. Random subsets with **5K**, **10K**, **30K**, and the full **110K speakers** show that more pretraining speakers consistently improve performance, and that larger models benefit more strongly from larger data. Model scaling from **ResNet34** through **ResNet293** also continues to improve performance, with no saturation reported in the tested range [2407.11510].

The final single-model result uses **SimAM-ResNet293** with post-processing. The reported numbers on VoxCeleb1-O are **0.23 EER / 0.013 minDCF** for the base model, **0.22 / 0.009** with **AS-Norm**, and **0.17 / 0.006** with **QMF**. The paper presents this as a new **single-model state-of-the-art** on VoxCeleb1-O [2407.11510].

These experiments also clarify what VoxBlink2 is not. It is not presented as a replacement for domain-aligned fine-tuning. Pretraining on **VB2 alone** yields **0.893 EER** and **0.093 minDCF\_{0.01}** on VoxCeleb1-O, worse than VC2-only pretraining. The best results come from **large-scale pretraining on VoxBlink2 followed by VC2 fine-tuning**, not from naive dataset substitution [2407.11510].

## 4. Open-Set Speaker Identification benchmark

VoxBlink2 is also the basis for the paper’s **Open-Set Speaker Identification** agenda. OSSI is framed as a more realistic extension of verification: the system must match a probe utterance to a known gallery speaker or reject it as unknown. The benchmark is built from **VoxBlink-clean**, specifically **1,028,095 utterances from 18,381 speakers**, and uses a gallery set \(G\) together with a probe set \(P = K \cup U\), where \(K\) are known queries and \(U\) are unknown queries [2407.11510].

Three protocol sizes are defined. **VB-Eval-S** uses **60** gallery speakers with **30** known and **30** unknown queries; **VB-Eval-M** uses **600 / 300 / 300**; **VB-Eval-L** uses **6,000 / 3,000 / 3,000**. Each protocol is further instantiated with **1**, **3**, or **5** enrollment utterances per speaker, yielding **9 evaluation protocols** in total [2407.11510].

The paper formalizes the evaluation with **DIR** and **FAR**. For threshold \(\theta\) and rank \(n\),
$$
\begin{aligned}
DIR(\theta,n) = \frac{\lvert{ rank(p)\geq n \wedge sim(p,G_p)\geq \theta;p\in K}\rvert}{\lvert K \rvert}
\end{aligned}
$$
and
$$
\begin{aligned}
FAR(\theta) = \frac{\lvert{ sim(p,G_p)\geq \theta;p\in U}\rvert}{\lvert U \rvert}.
\end{aligned}
$$
The reported benchmark uses **top-1** identification and reports **DIR@FAR** operating points [2407.11510].

The OSSI baseline uses a **ResNet50** backbone pretrained on **VB2**; the paper notes that this model obtains **1.02% EER on VoxCeleb1-O**. The open-set results show strong small-gallery performance but severe degradation at scale. For example, **VB-Eval-S** with **5** enrollments reaches **96.97** at **DIR@FAR=0.001**, whereas **VB-Eval-L** with **5** enrollments reaches only **24.94** at the same operating point. The general trends are explicit: more enrollment utterances improve DIR, but performance drops sharply as gallery size increases [2407.11510].

The OSSI benchmark is therefore one of VoxBlink2’s main conceptual contributions. It recasts large-scale speaker recognition as a **1:N plus rejection** problem rather than only a 1:1 verification problem. The paper also compares speaker and face modalities and concludes that both perform strongly on small galleries but both degrade on **VB-Eval-L**, which it interprets as evidence that very-large-scale open-set identification remains unsolved [2407.11510].

## 5. Use in NIST SRE24 open-condition systems

Later work makes VoxBlink2 operationally important in **NIST SRE24** open-condition systems. The ABC team describes VoxBlink2 as the central public corpus for the open condition, explicitly calling it the “largest publicly available speaker verification dataset” and using it to train **ResNet-152-VB** systems for conversational telephone speech after telephone-oriented adaptation. Their recipe includes **downsampling the audio to 8kHz** and applying **GSM codec to 50%** of the data via **Sox** to simulate the telephone channel, followed by fine-tuning on **CTS Superset**. With **40 s** fine-tuning segments, the reported **ResNet152-VB** system reaches **\(C_{primary}=0.491\)** and **EER \(=6.47\%\)** on **SRE24 eval** [2505.15320].

That paper is also explicit that VoxBlink2 is an **open-condition-only** resource. It is treated as a **large-scale out-of-domain dataset** for SRE24, and its value is tied to speaker diversity plus domain adaptation. The same paper notes that a hybrid recipe using original **16 kHz** data plus a telephone-simulated copy improves broad-domain generalization relative to a purely telephone-adapted version [2505.15320].

A second SRE24 example comes from the CL-UZH submission. There, VoxBlink2 appears only in the **open-condition** systems, not in the fixed condition. The open-condition description states that separate audio and visual networks were trained on **VoxBlink2 and VoxCeleb/VoxCeleb2**, with a **ResNet293** audio model and a visual model described in prose as **ResNet-based ArcFace**, while the result table labels the visual system **FaceNet**. The multimodal system concatenates audio and visual embeddings and applies **cosine distance** [2510.00952].

The CL-UZH numbers show the practical effect of VoxBlink2 as part of the open-condition training pool. In the open condition, **ResNet293, Audio** reports **EER 10.2 / 8.47** on **dev / eval**, and **ResNet+FN, AV** reports **EER 7.7 / 8.29** on **dev / eval**. The paper also states that the open-condition multimodal system “consistently outperform[ed] single-modality approaches,” especially in EER [2510.00952].

These SRE24 papers are useful for clarifying a common misunderstanding. They do not describe VoxBlink2 as a special blink-dynamics resource, nor do they report blink-specific preprocessing. In both cases, VoxBlink2 functions as a **training corpus for pretrained speaker or face encoders** in open-condition systems [2505.15320].

## 6. Limitations, interpretation, and data-governance issues

Despite its scale, VoxBlink2 has several explicit limitations. The paper does **not** provide a conventional train/dev/test split for VoxBlink2 itself; instead, VB2 is used for training, **VoxCeleb1** is used for speaker verification evaluation, and **VoxBlink-clean** is used for OSSI. It also does not report demographic breakdowns such as gender balance, age, nationality, ethnicity, or detailed language proportions, even though the retrieval pipeline spans **18 languages** and is described as globally diverse [2407.11510].

Quality estimation is also limited. The reported improvement from **72%** to **92%** label accuracy is based on manual inspection of only **50 randomly sampled speakers**. This establishes a direction of improvement, but not a comprehensive error characterization [2407.11510].

The paper additionally implies several governance concerns. VoxBlink2 is mined from **YouTube users**, using both **avatars** and uploaded videos, which raises issues of consent, reuse of public personal data, biometric privacy, and misuse risk. These concerns are implicit in the data source and acknowledged in the paper’s caveat section, but there is no extended fairness audit, no detailed ethics section, and no exhaustive governance analysis [2407.11510].

From an evaluation standpoint, the largest OSSI protocol remains difficult. Even with **5** enrollment utterances per speaker, **VB-Eval-L** reaches only **24.94** at **DIR@FAR=0.001**. This suggests that VoxBlink2’s importance lies not only in improved verification pretraining but also in exposing an open-set identification regime that remains substantially unsolved [2407.11510].

Taken together, the available evidence supports a precise characterization. VoxBlink2 is a **100K+ speaker audio-visual corpus** that substantially extends VoxBlink in scale, temporal coverage, and annotation purity; it supports strong speaker-verification pretraining when combined with domain-aligned fine-tuning; it introduces an open-set identification benchmark based on **DIR@FAR**; and it has already been adopted as a core open-condition training resource in SRE24 systems [2407.11510].

Source: https://www.emergentmind.com/topics/voxblink2