---
title: 'Speechocean762: L2 Speech Corpus for CAPT'
url: https://www.emergentmind.com/topics/speechocean762
type: topic
---

# Speechocean762: L2 Speech Corpus for CAPT

Speechocean762, introduced as “speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessment,” is an open-source L2 English read-speech corpus designed specifically for pronunciation assessment. It contains 5,000 English utterances from 250 non-native speakers whose mother tongue is Mandarin, with half of the speakers being children, and it provides expert annotations at sentence, word, and phoneme levels. Because it combines multi-granularity labels, public availability, and an accompanying Kaldi baseline, it has become a canonical benchmark in computer-assisted pronunciation training (CAPT), automatic pronunciation assessment (APA), and mispronunciation detection and diagnosis (MDD) research [2104.01378, 2601.16230].

## 1. Corpus design, composition, and recording conditions

The corpus was created to support pronunciation assessment and broader computer-assisted language learning workflows with a freely downloadable, manually annotated, non-native English benchmark. It comprises 5,000 English utterances from 250 learners, with 20 sentences read by each speaker. The speaker population is balanced at the adult/child level, with half of the speakers being children and half adults; gender is reported as 1:1 for both adults and children. The total audio duration is about 6 hours [2104.01378].

Speechocean762 is a read-aloud corpus rather than a spontaneous-speech collection. The prompts were selected from daily-life text and contain about 2,600 common English words. This design makes the corpus especially suitable for text-conditioned CAPT pipelines, forced alignment, GOP-style feature extraction, and prompt-aware multimodal assessment [2104.01378, 2305.11438].

The original paper reports controlled but lightweight recording conditions. Speech was recorded in a quiet room of approximately \(3\times 3\) meters; speakers held mobile phones approximately 20 cm from the mouth; and the devices included popular Apple, Samsung, Xiaomi, and Huawei models. The paper does not report sampling rate, bit depth, channel count, or file format [2104.01378].

The corpus is organized with a random speaker-balanced split of 125 speakers for training and 125 speakers for testing. Later benchmark papers typically follow this 2,500/2,500 utterance partition, although zero-shot studies sometimes evaluate all 5,000 utterances directly because no corpus-specific training is required [2104.01378, 2601.16230].

## 2. Annotation protocol and scoring semantics

Five experts scored each utterance independently under the same metrics. The annotation workflow is unusually explicit about canonical phone selection: when a word has multiple acceptable canonical phone sequences, the experts first inspect the alternatives, select the closest sequence, and then vote to determine the final canonical sequence; that chosen sequence is included as corpus meta-information. During scoring, experts read the transcript and listen repeatedly at least three times, and they enter scores through the “SpeechOcean uTrans” application, which performs consistency checks across levels [2104.01378].

The corpus provides phoneme-level, word-level, and sentence-level labels. The original scoring definitions are as follows:

| Level | Aspect | Scale or definition |
|---|---|---|
| Phoneme | Accuracy | 2 correct; 1 heavy accent; 0 incorrect or missed |
| Word | Accuracy | 10 correct; 7–9 heavy accent; 4–6 \(\le 30\%\) phones wrong; 2–3 \(>30\%\) wrong or another word; 0–1 hard to distinguish or missed |
| Word | Stress | 10 correct or monosyllable; 5 incorrect |
| Sentence | Accuracy | 9–10 excellent; 7–8 good with a few mispronunciations; 5–6 many but understandable; 3–4 awkward with many serious mispronunciations; 0–2 unintelligible or no voice |
| Sentence | Fluency | 8–10 coherent; 6–7 generally coherent with a few pauses/repetition/stammering; 4–5 incoherent with many; 0–3 not able to read as a whole or no voice |
| Sentence | Prosody | 9–10 correct intonation and stable rhythm; 7–8 nearly correct intonation at stable speed; 3–6 unstable speed or inappropriate intonation; 0–2 too stammering or no voice |
| Sentence | Completeness | Percentage of target words actually pronounced |

The original paper states that sentence-level scores vary across 3 to 10, while most word-level and phoneme-level scores fall in the range 8 to 10 when phoneme scores are linearly mapped to a 0–10 plot for visualization. It does not report an aggregation rule across the five raters, nor inter-annotator agreement metrics such as kappa or ICC [2104.01378].

Later benchmark work has exposed an important property of the label distribution. Completeness is repeatedly described as ambiguous and highly skewed: one study reports that human completeness ratings are strongly biased toward the maximum score of 10, even when content diverges, while another reports that only 8 training utterances have completeness scores in the range 5–8 and only 14 test utterances have completeness scores between 0–8. This makes completeness unusually difficult for correlation-based evaluation [2601.16230, 2509.15701].

A further convention emerged in later work: word- and sentence-level labels are often rescaled from 0–10 to 0–2 for joint modeling with phoneme accuracy, or conversely phoneme labels are linearly mapped from 0–2 to 0–10 for uniform prompting and reporting. Benchmark papers also frequently add “word total” and “utterance total” as supervised targets, even though the original manual-scoring table emphasizes the component aspects rather than a separate total rubric [2205.03432, 2509.15701].

## 3. Availability, official baseline, and reproducibility infrastructure

Speechocean762 is distributed through OpenSLR as ID 101 and is explicitly allowed to be used freely for commercial and non-commercial purposes. The original release also provides an open-source Kaldi baseline recipe at `egs/gop_speechocean762`, making the corpus unusually reproducible by CAPT standards [2104.01378].

The official baseline demonstrates a classical GOP workflow. A native acoustic model trained on LibriSpeech is used to generate frame-level posteriors, forced alignment is performed against the expert-voted canonical phone sequences, GOP values and GOP-based features are computed, and per-phone regressors are trained to predict phoneme scores. The primary regressor is an SVR, with polynomial regression on raw GOP values offered as a lighter alternative [2104.01378].

A technically important design choice in the Kaldi recipe is direct construction of the LG FST from the expert-selected canonical phone sequences. This avoids the mismatch that would arise if alignment were built from a lexicon containing multiple pronunciations while the manual labels were tied to a single expert-voted sequence. The recipe also addresses score imbalance by supplementing each phone regressor with high-score samples from other phones, randomly relabeled as zero for the current phone [2104.01378].

The original baseline reports modest but serviceable phoneme-level performance: using the GOP value directly yields MSE \(=0.69\) and PCC \(=0.25\), while using GOP-based features improves this to MSE \(=0.16\) and PCC \(=0.45\) [2104.01378].

Later work standardized the official split even further. One influential benchmark paper reports that the training set contains 2,500 utterances, 15,849 words, and 47,076 phones, while the test set contains 2,500 utterances, 15,967 words, and 47,369 phones. This helped establish a consistent public protocol for cross-paper comparison [2205.03432].

## 4. Benchmark roles across APA, fluency assessment, MDD, and multimodal scoring

Speechocean762 supports several distinct but overlapping research programs. In multi-aspect, multi-granularity APA, GOPT treats the corpus as a joint regression problem over phoneme accuracy, word accuracy/stress/total, and utterance accuracy/fluency/completeness/prosody/total using GOP features from a public Librispeech acoustic model [2205.03432]. Subsequent systems changed either the feature stack or the inductive bias: 3M augments GOP with duration, energy, and SSL features plus a vowel/consonant positional embedding [2208.09110]; HiPAMA introduces sup-phonemes, depth-wise separable convolution, and score-restraint attention pooling [2305.18146]; Acoustic Feature Mixup explicitly targets score imbalance, especially for Stress and Completeness [2406.15723]; and JCAPT combines multi-granular APA with phone-level MDD in a Mamba-based joint framework with phonological features and think tokens [2506.19315].

The corpus is also a principal benchmark for fluency-specific work. A phonetic and prosody-aware SSL approach uses phone identities, durations, and forced-alignment-derived deep features, then fine-tunes on utterance-level fluency scores [2305.11438]. CBF-AFA reformulates fluency into a three-class problem—Low\_fluency \(0\!-\!5\), Medium\_fluency \(6\!-\!7\), and High\_fluency \(8\!-\!10\)—and applies Silero-VAD, breath-group chunking, multi-SSL fusion, and a CNN-BiLSTM over chunk sequences [2506.20243].

Zero-shot and resource-light assessment lines use Speechocean762 to test how much pronunciation information can be extracted without score supervision. A HuBERT-based masked token recovery method performs sentence-level APA without ASR, text references, or fine-tuning [2305.19563]. A multimodal LLM system based on Data2Vec2, an m-adapter, and a frozen Qwen-7B predicts sentence-level accuracy and fluency from audio plus prompt text [2407.09209]. A later lightweight framework discretizes speech with HuBERT and K-means, models native token surprisal with a 3-gram LM, and adds transcript-guided Text2DUnit--DTW alignment features trained only on native speech resources [2606.19910].

SpeechLLM and LMM studies have further expanded the benchmark’s role. Qwen2-Audio-7B-Instruct has been evaluated zero-shot for sentence-level accuracy, fluency, prosody, and completeness [2601.16230]. GPT-4o has been used in an align-free zero-shot setup for multi-level scoring and feedback generation [2503.11229]. Fine-tuned multimodal systems include a LoRA-adapted Phi-4 model for joint APA and MDD [2509.02915], a fine-tuned Qwen2-Audio model with SimPO or DPO-style preference learning for multi-granular APA [2509.15701], and a rubric-guided SpeechLLM that jointly predicts sentence-, word-, and phoneme-level labels while generating natural-language rationales [2606.09470].

MDD and phone-level pronunciation scoring remain equally central. CoCA-MDD uses coupled cross-attention between acoustic and reference-phone streams and reports phone-level scoring on Speechocean762 [2111.08191]. SpeechBlender augments phoneme-level training data by smoothly blending donor and candidate phone segments selected from confusion pairs [2211.00923]. L1-MultiMDD treats Speechocean762 as an unseen English-by-Mandarin test set for multilingual zero-shot generalization, including both adults and children [2309.07719]. Substitution-aware alignment-free GOP constrains phoneme substitutions through a handcrafted phoneme confusion map and evaluates all 5,000 utterances without relying on forced alignment in the scoring path [2506.02080].

## 5. Representative empirical results

Reported numbers on Speechocean762 are highly task-dependent and are not directly comparable across phoneme regression, sentence-level APA, three-class fluency classification, and MDD. Even so, a few results have become reference points for the benchmark:

| Setting | Representative reported result | Source |
|---|---|---|
| Original Kaldi baseline | Phoneme-level PCC \(=0.45\), MSE \(=0.16\) with GOP-based features | [2104.01378] |
| GOPT | Phoneme PCC \(=0.612\); utterance total PCC \(=0.742\) | [2205.03432] |
| Prosody-aware SSL fluency scoring | Best fluency PCC \(=0.838\) with BLSTM-pre and duration-only reconstruction | [2305.11438] |
| Zero-shot HuBERT APA | Sentence-level PCC \(=0.60\); supervised GoP baseline \(=0.64\); GOPT \(=0.74\) | [2305.19563] |
| Chunk-based fluency classification | F1 \(=0.825\), PCC \(=0.796\) | [2506.20243] |
| Joint APA+MDD with JCAPT | Phoneme PCC \(=0.720\); utterance total PCC \(=0.806\); MDD F1 \(=51.05\%\) | [2506.19315] |
| Native-resource-only discrete token surprisal with transcript guidance | Accuracy/Fluency/Prosody PCC \(=0.661/0.763/0.753\) | [2606.19910] |

Zero-shot speech-LLM behavior on the corpus is especially revealing. Qwen2-Audio-7B-Instruct achieves within-\(\pm 2\) agreement of \(87.4\%\), \(89.5\%\), and \(87.5\%\) for accuracy, fluency, and prosody, but the raw PCCs are only \(0.206\), \(0.053\), and \(0.140\), while completeness drops to \(-0.021\). The same study reports a systematic overestimation bias for low-quality speech, including \(0\%\) low-score coverage for utterances with ground-truth accuracy \(\le 6\) [2601.16230].

Fine-tuned multimodal models narrow that gap but do not remove it uniformly across granularities. A LoRA-tuned Phi-4 multimodal model trained on Speechocean762 reports sentence-level PCCs up to \(0.743\) for accuracy, \(0.733\) for fluency, and \(0.714\) for prosodic, while achieving WER \(=0.14\) and PER \(=0.114\) in joint APA/MDD evaluation [2509.02915]. A fine-tuned Qwen2-Audio model reaches competitive word- and sentence-level results, but its phoneme-level PCC remains \(0.38\) in multi-granularity experiments, below specialized baselines such as GOPT and HMamba [2509.15701].

At the phone level, MDD-oriented methods also show the corpus’s diagnostic range. CoCA-MDD reports phone-level PCC \(=0.58\), improving over a GOP-NN + SVR baseline at \(0.45\) [2111.08191]. SpeechBlender raises phoneme-level PCC to \(0.63\) with MSE \(=0.085\) through augmentation [2211.00923]. In alignment-free GOP, the PP-AF GOP UPS variant reports AUC \(=0.916\), MCC \(=0.395\), and phoneme-score PCC \(=0.502\) under the “high conf” setting [2506.02080].

## 6. Strengths, caveats, and continuing significance

Speechocean762 has several properties that explain its longevity. It is open-source and free for both commercial and non-commercial use; it includes both children and adults; it provides sentence-, word-, and phoneme-level annotations from five experts; it stores expert-voted canonical phone sequences; and it ships with a public Kaldi baseline recipe [2104.01378]. A plausible implication is that few public CAPT corpora offer as clean a bridge between classical forced-alignment pipelines, SSL-based assessment, zero-shot methods, and modern multimodal generative models.

At the same time, several limitations recur throughout the literature. The corpus is Mandarin-L1 only, so cross-L1 generalization must be established elsewhere or through external zero-shot testing [2104.01378, 2309.07719]. It is also a read-aloud benchmark with known prompts, which strongly favors text-conditioned assessment and may underrepresent open-response or spontaneous-speech phenomena [2104.01378, 2407.09209]. The original release does not report the aggregation rule across five raters, inter-annotator agreement metrics, or detailed audio specifications such as sampling rate and bit depth [2104.01378].

Label imbalance is a second persistent issue. Later studies repeatedly identify severe skew in Completeness and, to a lesser degree, Stress or low-score regions more generally [2601.16230, 2406.15723, 2509.15701]. This suggests that reported gains on those aspects should be interpreted together with the task formulation, the metric choice, and the score transformation used in each paper. The same caveat applies to phoneme labels: the native phoneme scale is 0–2, but many later systems rescale it to 0–10 for unified prompting or joint training, which changes the numerical regime of evaluation without altering the underlying annotation granularity [2509.15701].

A further pattern is selective use of the corpus. Many studies exploit only sentence-level labels, only fluency, or only phoneme accuracy, even though the dataset was designed as a multi-granularity benchmark [2305.11438, 2305.19563]. This reflects both the richness of the corpus and the engineering difficulty of using all annotation levels simultaneously.

Despite these caveats, Speechocean762 remains active well into 2026 as a sole or primary public benchmark for joint multi-granular assessment, speech-LLM rationales, and native-resource-only pronunciation scoring [2606.09470, 2606.19910]. Its continuing relevance lies in a rare combination: public access, controlled prompts, expert multi-level labels, and a benchmark history broad enough to connect GOP-era CAPT, SSL-based assessment, alignment-free methods, and instruction-following multimodal models within a single shared evaluation setting.

Source: https://www.emergentmind.com/topics/speechocean762