---
title: WSJ0-2mix Dataset for Speech Separation
url: https://www.emergentmind.com/topics/wsj0-2mix-dataset
type: topic
---

# WSJ0-2mix Dataset for Speech Separation

The WSJ0-2mix dataset is the prevailing benchmark for single-channel speech separation and related tasks such as target-speaker automatic speech recognition (TS-ASR). Constructed from the Wall Street Journal 0 (WSJ0) read-speech corpus, it enables reproducible evaluation of algorithms that aim to disentangle overlapping utterances from two distinct speakers under strictly controlled acoustic conditions. Over time, WSJ0-2mix has been extended to create more challenging variants (e.g., with additive noise or multiple interferers), and repurposed for downstream ASR, perceptual quality, and codec-based separation research. Its protocol, split structure, and reference ambiguities have progressively shaped methodologies and evaluation standards for speech separation.

## 1. Corpus Construction and Mixture Generation

WSJ0-2mix is created by drawing utterance pairs from the WSJ0 corpus, recorded with close-talk microphones in acoustically controlled settings. For each mixture, two distinct speakers are randomly selected, and a single utterance from each is chosen. The mixture is generated in the time domain as:

$$
x[n] = s_1[n] + \alpha\,s_2[n]
$$

where $s_1[n]$ and $s_2[n]$ are the clean reference waveforms, and the gain factor $\alpha$ is computed to achieve a target signal-to-noise ratio (SNR), randomly sampled from a uniform distribution (commonly $\mathcal{U}(0,5)$ dB). The precise formula for $\alpha$ is:

$$
\alpha = \frac{\|s_1\|_2}{\|s_2\|_2} \cdot 10^{-\mathrm{SNR}/20}
$$

depending on the chosen SNR and the source energies [2308.05218, 1907.01160, 1905.03500].

Two main mixing strategies are provided:

-**Min** variant: The output mixture is truncated to the length of the shorter utterance, ensuring near-complete temporal overlap (overlap ratio ≃ 100%).  
-**Max** variant: The shorter utterance is padded with silence to match the length of the longer, aligned at onset, leading to possible non-overlapping trailing segments [1907.01160].

Recent repurposings (e.g., WSJ0-2mix-extr [2308.05218], see section 3) require additional logic such as auxiliary utterance selection or silence padding for temporal alignment.

## 2. Dataset Splits and Extensions

WSJ0-2mix adopts strictly speaker-disjoint splits. The canonical configuration is:

| Split     | #Mixtures | Duration (h) | Sources     |
|-----------|-----------|--------------|-------------|
| Train     | 20,000    | ~30          | si_tr_s     |
| Validation| 5,000     | ~10          | si_dt_05    |
| Test      | 3,000     | ~5           | si_et_05    |

Speakers do not overlap between splits. Each mixture is an independent random pairing; utterance durations range from approximately 1–8 seconds. The “extr” extension (WSJ0-2mix-extr) appends a third, auxiliary utterance from the target speaker, supporting explicit target-speaker modeling in TS-ASR regimes [2308.05218].

Multi-speaker expansions (e.g., WSJ0-3mix-extr) follow an analogous procedure using three utterances from three speakers, creating mixtures with two interferers for robustness assessment [2308.05218, 2001.00391].

## 3. Preprocessing and Feature Extraction

The dataset is distributed at both 8 kHz and 16 kHz sample rates [1907.01160, 2411.17998]. All recordings are free from artificially added noise or reverberation (“clean”), although subsequent studies frequently simulate reverberant conditions via convolution with room impulse responses (RIRs) [2001.00391].

Feature extraction protocols vary by downstream task:

- **ASR and source separation**: 80-dimensional log-Mel filterbanks (25 ms window, 10 ms hop) are standard; SpecAugment and volume/speed perturbation are applied for data augmentation [2308.05218].
- **Codec-based approaches**: If model frontends require a higher sample rate, mixtures and references are resampled appropriately (e.g., to 16 kHz for neural audio codecs) [2411.17998].
- **“Far-field” variants**: Clean waveforms are convolved with randomly synthesized RIRs, yielding multi-channel, reverberant mixtures [2001.00391].
- **Sparsely overlapping variants**: Silence is inserted between utterances to modulate overlap ratio, with per-gap energy normalization to preserve perceptual continuity [1905.03500].

Explicit voice-activity detection is generally not performed in the standard pipeline.

## 4. Evaluation Protocols, Metrics, and Benchmarks

WSJ0-2mix is evaluated primarily in two regimes: generic source separation and target-speaker ASR.

### Standard Metrics

- **Scale-Invariant Signal-to-Distortion Ratio (SI-SDR)** and **SDR** (BSS_EVAL): Intrusive measures used to quantify separation fidelity. SI-SDR has become preferred due to invariance to scaling [2001.00391, 2508.14623].
- **Word Error Rate (WER)**: For ASR tasks, WER is computed after decoding separated streams, using standard n-gram language models [2308.05218, 1905.03500].
- **Perceptual metrics**: DNSMOS, STOI, and PESQ are employed to measure speech intelligibility and quality, especially for codec or end-user evaluation scenarios [2411.17998].

### Results and Comparative Baselines

Recent results on WSJ0-2mix(-extr) exhibit rapid advances:

| Model                       | Loss       | 2-mix WER | Year      | Reference    |
|-----------------------------|------------|-----------|-----------|--------------|
| Conformer-CTC (1spk)        | CTC        |   36.7%   | 2023      | [2308.05218] |
| SpeakerBeam                 | x-ent      |   30.6%   | 2019      | [2308.05218] |
| Exformer+Conformer-CTC      | Si-SiNR+CTC|   13.2%   | 2023      | [2308.05218] |
| CONF-TSASR (CTC+Spec)       | CTC+Spec   |   4.2% ★  | 2023      | [2308.05218] |
| DPCL+DNN-HMM                |            |  16.5%    | 2019      | [1905.03500] |

★ Denotes new state of the art as of the respective publication.

Codec-based models evaluated on WSJ0-2mix achieve DNSMOS-OVRL in the range 1.73–1.89 and STOI values 0.74–0.83, varying by loss type and codec [2411.17998].

## 5. Variants and Related Datasets: WHAM! and Noisy/Enhanced References

The WHAM! extension overlays real ambient noise recordings onto WSJ0-2mix mixtures, sampled from San Francisco cafés, parks, and offices [1907.01160]. Noise segments are precisely selected to avoid foreground speech and matched in duration to the underlying mixture. The noise is added at an SNR sampled from $\mathcal{U}(-6, 3)$ dB, substantially increasing separation difficulty. Both “min” and “max” overlap configurations are available at 8 kHz and 16 kHz. SI-SDR performance drops by 1–2 dB relative to clean WSJ0-2mix, illustrating the increased challenge [1907.01160, 2508.14623].

A salient finding is that WSJ0-2mix's “clean” references sometimes contain low-level noise and recording artifacts. This latent noise introduces a ceiling effect in SI-SDR—no algorithm can surpass the actual SNR of the reference, and models are incentivized to pass through noise if SI-SDR is used for training [2508.14623]. Approaches that enhance references (e.g., via MetricGAN+) and reconstruct mixtures can yield perceptually cleaner output but may introduce artifacts, notably in Colouration and Discontinuity MOS. Negative correlation between SI-SDR and perceived noisiness is observed across models [2508.14623].

## 6. Applications, Generalizations, and Research Impact

WSJ0-2mix serves as a canonical testbed for deep clustering, mask-inference, time-domain separation (e.g., Conv-TasNet, SepFormer), TS-ASR, and codec-based separation. Extensions to three-speaker mixtures (WSJ0-3mix), far-field/multi-channel arrangements [2001.00391], and overlapping ratio control [1905.03500] facilitate controlled exploration of separation in realistic and adversarial scenarios. The dataset's rigorously defined splits and construction protocol have standardized comparative evaluation, shaped the development of permutation-invariant training objectives, and exposed critical methodology gaps (e.g., reference noisiness and SI-SDR limitations).

A plausible implication is that future research should prioritize development of cleaner reference corpora and non-intrusive evaluation metrics, given the demonstrated limitations of SI-SDR as both loss and metric in the presence of reference noise [2508.14623].

## 7. Common Misconceptions and Methodological Caveats

A prominent misconception is that WSJ0-2mix is acoustically pristine and thus suitable for arbitrary separation/denoising objectives. However, intrinsic recording noise in the WSJ0 corpus constrains maximum achievable SI-SDR, biases model outputs toward reference noise when SI-SDR is used for optimization, and complicates fair benchmarking [2508.14623]. Another misconception concerns overlap: the standard WSJ0-2mix mixtures are fully overlapped except for minor silence frames. Realistic conversational or meeting scenarios (partial overlap, far-field, multi-speaker) are not directly represented and require simulation or alternative benchmarks [1905.03500, 2001.00391].

In summary, WSJ0-2mix is a foundational speech separation resource with meticulously defined protocols and a diverse ecosystem of extensions. Its ongoing usage continues to inform both fundamental and applied research in source separation and robust speech recognition.

Source: https://www.emergentmind.com/topics/wsj0-2mix-dataset