---
title: VCTK-2mix Speech Separation Test Set
url: https://www.emergentmind.com/topics/vctk-2mix
type: topic
---

# VCTK-2mix Speech Separation Test Set

VCTK-2mix is a two-speaker mixture dataset derived from the CSTR VCTK corpus of read English speech. It is designed as a test-only benchmark to evaluate the generalization ability of single-channel speech separation models, particularly when these models are trained on different domains or speaker populations. VCTK-2mix has emerged as a critical out-of-distribution evaluation set, supplementing established corpora such as WSJ0-2mix, LibriMix, and WHAM!, by representing speech with broader accent diversity and more conversational scenarios [2002.08688][2005.11262]. The dataset is used extensively for benchmarking time-domain separation architectures, notably Conv-TasNet and its enhanced variants.

## 1. Dataset Construction Protocols

VCTK-2mix is prepared from the CSTR VCTK corpus, which includes recordings from 108–109 native English speakers of varying genders and accents. The construction process comprises the following steps:

- **Speaker Selection and Preprocessing:** All available speakers are included; extended silences are removed using an energy-based VAD (threshold: 20 dB), resulting in 108 speakers in the final set [2005.11262]. No gender or accent stratification is imposed.
  
- **Speaker Pairing and Mixing Procedure:** For each mixture, two utterances from different speakers are randomly sampled and loudness-normalized. In one protocol, SNR for mixing is sampled uniformly between –5 dB and +5 dB [2002.08688]; in another, according to LUFS (ITU-R BS.1770-4), with speech and noise loudness sampled as $U_s \sim \mathrm{Uniform}(-33, -25)\,\mathrm{LUFS}$ and $U_n \sim \mathrm{Uniform}(-38, -30)\,\mathrm{LUFS}$, respectively [2005.11262].

- **Mixing Equation:** The mixture is created as $x = s_1 + \alpha s_2$, with $\alpha$ selected such that $10 \log_{10} \frac{\|s_1\|^2}{\|\alpha s_2\|^2} = \mathrm{SNR}$ [2002.08688].

- **Temporal Alignment and Modes:** Both sources are fully overlapped (synchronous start). Mixtures are generated in both "min" (truncate to shortest utterance) and "max" (zero-pad to longest) modes, with audio resampled at either 8 kHz [2002.08688] or 16 kHz [2005.11262].

- **Dataset Split and Size:** VCTK-2mix is strictly a test set, with no training or validation splits. Sets contain either 3,000 [2005.11262] or 4,000 [2002.08688] mixtures. Each mixture is 4 s long in [2002.08688], yielding approximately 4.44 hours of evaluation data.

- **File Structure:** The directory structure and file naming are identical to LibriMix, with zero-padded indices and per-source reference tracks.

A summary table of construction parameters from major protocols follows:

| Protocol Source   | # Mixtures | Speakers | Sampling Rate | SNR Sampling               | Modes           |
|-------------------|------------|----------|---------------|----------------------------|-----------------|
| [2002.08688]      | 4,000      | Not specified | 8 kHz         | Uniform(–5, +5) dB         | Only max        |
| [2005.11262]      | 3,000      | 108      | 16 kHz, 8 kHz  | LUFS-based as above        | min, max        |

## 2. Objective Evaluation Metrics

VCTK-2mix benchmarking primarily uses scale-invariant signal-to-noise ratio improvement (SI-SNRᵢ) as the main metric, alongside (optionally) source-to-distortion ratio improvement (SDRᵢ).

- **Scale-Invariant SNR (SI-SNR):**
  
  $$
  \begin{align*}
  \alpha &= \frac{\langle \hat{s}, s \rangle}{\|s\|^2} \\
  s_\mathrm{target} &= \alpha s \\
  e_\mathrm{noise} &= \hat{s} - s_\mathrm{target} \\
  \mathrm{SI-SNR}(\hat{s}, s) &= 10 \log_{10} \frac{\|s_\mathrm{target}\|^2}{\|e_\mathrm{noise}\|^2}
  \end{align*}
  $$
  [2002.08688][2005.11262]

- **SI-SNR Improvement:**
  
  $$
  \mathrm{SI\text{-}SNR}_i = \mathrm{SI\text{-}SNR}(\hat{s}, s) - \mathrm{SI\text{-}SNR}(x, s)
  $$
- **Source-to-Distortion Ratio (SDR):** Defined with BSS-Eval conventions; not always reported for VCTK-2mix [2005.11262].

Perceptual metrics such as PESQ and STOI are not reported for VCTK-2mix in the surveyed studies.

## 3. Use in Benchmarking and Experimental Protocols

VCTK-2mix's role is as an out-of-distribution (OOD) test set for models trained on other speech separation corpora (WSJ0-2mix, LibriMix, LibriTTS):

- **No Training/Validation:** VCTK-2mix is never used for fitting or tuning hyperparameters.
- **Cross-Dataset Generalization:** The set quantifies degradation in SI-SNRᵢ when source separation models are tested outside their training distribution.
- **Mixture Types:** Both clean (2spk-C, speech only) and noisy (2spk-N, speech + WHAM! noise) mixtures are available [2005.11262].

This benchmarking context is critical for advancing speech separation systems towards domain robustness and real-world applicability.

## 4. Empirical Results and Model Generalization

Key experimental findings across major studies consistently show significant generalization gaps when models are evaluated on VCTK-2mix versus their training corpus:

- **Conv-TasNet Trained on WSJ0-2mix:** SI-SNRᵢ ≈ 15.4 dB on in-domain test, but ≈ 9.1 dB on VCTK-2mix (Δ ≈ –6.3 dB) [2002.08688].
- **Conv-TasNet Trained on LibriTTS:** SI-SNRᵢ ≈ 17–17.5 dB in-domain, but 9.8–11.5 dB on VCTK-2mix, depending on encoder/decoder depth and loss [2002.08688].
- **Conv-TasNet Trained on LibriMix Train-360 (16 kHz):** SI-SNRᵢ ≈ 12.1 dB (clean), ≈10.8 dB (noisy) [2005.11262].
- **Conv-TasNet Trained on WHAM!:** SI-SNRᵢ ≈ 8.2 dB to ≈ 6.8 dB on VCTK-2mix; a further drop with noisy mixtures.
- **Effect of Enhanced Architectures:** Deep (non-linear, multi-layer) encoder/decoder yields 0.3–1.0 dB gain; addition of an augmented "power-law" loss term yields up to 1.7 dB gain [2002.08688].

Performance drops of 4–7 dB SI-SNRᵢ relative to in-domain test sets are typical, reflecting the challenge of robust speech separation in real-world speaker conditions.

## 5. Architectural and Training Recommendations

Research leveraging VCTK-2mix highlights several protocols and design enhancements to improve cross-corpus robustness:

- **Encoder/Decoder Complexity:** Increasing the depth and non-linearity of the (time-domain) encoder/decoder (e.g., via stacked 1-D convolutional layers with PReLU or GLU activations, with or without dilation) enhances generalizability [2002.08688].
- **Augmented Loss Functions:** Combining SI-SNR with a power-law magnitude loss in the STFT domain ($\| |\mathrm{STFT}(\hat{s})|^{\alpha} - |\mathrm{STFT}(s)|^{\alpha} \|_1$, with $\alpha=0.5$, $\beta=0.01$) stabilizes training and further improves SI-SNRᵢ both in- and out-of-domain [2002.08688].
- **Training Data Diversity:** Larger speaker pools, expanded vocabulary, and more varied recording conditions in LibriMix/LibriTTS translate to smaller generalization error on VCTK-2mix compared to systems trained on WSJ0-2mix/WHAM! [2005.11262].
- **Loudness Normalization:** LUFS-based loudness normalization yields more realistic audio level variability than simple power-scaling, improving robustness [2005.11262].

The following procedural recommendations derive from these empirical findings:

1. Use LibriMix train-360 (clean, noisy) for training to maximize generalization.
2. Adopt LUFS-based level normalization in data preparation.
3. Apply noise speed-perturbation to further diversify noise instances.
4. Use SI-SNR loss (possibly augmented) and standardized evaluation splits.

## 6. Directory Structure and Benchmarking Best Practices

VCTK-2mix's distribution closely follows that of LibriMix for reproducibility and interoperability with existing speech separation pipelines. Directory structure is as follows (at 16 kHz, max-mode):

```
VCTK-2mix/
  └─ test/
      ├─ 2spk-C/
      │   ├─ mixture/
      │   ├─ s1/
      │   └─ s2/
      └─ 2spk-N/
          ├─ mixture/
          ├─ s1/
          └─ s2/
```
File naming is uniform: `mixture_0001.wav`, `s1_0001.wav`, `s2_0001.wav`, etc.

Benchmarking is performed on both clean (2spk-C) and noise-added (2spk-N) modes, using standardized evaluation scripts and segmentations. Observed generalization errors on VCTK-2mix should be reported alongside results for WSJ0-2mix and LibriMix to provide a comprehensive view of system robustness.

---

VCTK-2mix serves as a reproducible, challenging cross-dataset evaluation set for single-channel speech separation. Its diversity and public availability make it a principal resource for reporting generalization performance and for guiding future methods in architecture design, dataset construction, and loss formulation [2002.08688][2005.11262].

Source: https://www.emergentmind.com/topics/vctk-2mix