Speechocean762: L2 Speech Corpus for CAPT

• Speechocean762 is a publicly available non-native English read-speech corpus with 5,000 utterances from 250 Mandarin speakers, designed for pronunciation assessment.
• It offers multi-granularity annotations at sentence, word, and phoneme levels, enabling forced alignment, GOP feature extraction, and comprehensive CAPT evaluations.
• Accompanied by an open-source Kaldi baseline recipe, the corpus serves as a benchmark for automatic pronunciation assessment, fluency scoring, and mispronunciation detection research.

Speechocean762, introduced as “speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessment,” is an open-source L2 English read-speech corpus designed specifically for pronunciation assessment. It contains 5,000 English utterances from 250 non-native speakers whose mother tongue is Mandarin, with half of the speakers being children, and it provides expert annotations at sentence, word, and phoneme levels. Because it combines multi-granularity labels, public availability, and an accompanying Kaldi baseline, it has become a canonical benchmark in computer-assisted pronunciation training (CAPT), automatic pronunciation assessment (APA), and mispronunciation detection and diagnosis (MDD) research (Zhang et al., 2021, Parikh et al., 20 Jan 2026).

1. Corpus design, composition, and recording conditions

The corpus was created to support pronunciation assessment and broader computer-assisted language learning workflows with a freely downloadable, manually annotated, non-native English benchmark. It comprises 5,000 English utterances from 250 learners, with 20 sentences read by each speaker. The speaker population is balanced at the adult/child level, with half of the speakers being children and half adults; gender is reported as 1:1 for both adults and children. The total audio duration is about 6 hours (Zhang et al., 2021).

Speechocean762 is a read-aloud corpus rather than a spontaneous-speech collection. The prompts were selected from daily-life text and contain about 2,600 common English words. This design makes the corpus especially suitable for text-conditioned CAPT pipelines, forced alignment, GOP-style feature extraction, and prompt-aware multimodal assessment (Zhang et al., 2021, Fu et al., 2023).

The original paper reports controlled but lightweight recording conditions. Speech was recorded in a quiet room of approximately $3\times 3$ meters; speakers held mobile phones approximately 20 cm from the mouth; and the devices included popular Apple, Samsung, Xiaomi, and Huawei models. The paper does not report sampling rate, bit depth, channel count, or file format (Zhang et al., 2021).

The corpus is organized with a random speaker-balanced split of 125 speakers for training and 125 speakers for testing. Later benchmark papers typically follow this 2,500/2,500 utterance partition, although zero-shot studies sometimes evaluate all 5,000 utterances directly because no corpus-specific training is required (Zhang et al., 2021, Parikh et al., 20 Jan 2026).

2. Annotation protocol and scoring semantics

Five experts scored each utterance independently under the same metrics. The annotation workflow is unusually explicit about canonical phone selection: when a word has multiple acceptable canonical phone sequences, the experts first inspect the alternatives, select the closest sequence, and then vote to determine the final canonical sequence; that chosen sequence is included as corpus meta-information. During scoring, experts read the transcript and listen repeatedly at least three times, and they enter scores through the “SpeechOcean uTrans” application, which performs consistency checks across levels (Zhang et al., 2021).

The corpus provides phoneme-level, word-level, and sentence-level labels. The original scoring definitions are as follows:

Level	Aspect	Scale or definition
Phoneme	Accuracy	2 correct; 1 heavy accent; 0 incorrect or missed
Word	Accuracy	10 correct; 7–9 heavy accent; 4–6 $\le 30\%$ phones wrong; 2–3 $>30\%$ wrong or another word; 0–1 hard to distinguish or missed
Word	Stress	10 correct or monosyllable; 5 incorrect
Sentence	Accuracy	9–10 excellent; 7–8 good with a few mispronunciations; 5–6 many but understandable; 3–4 awkward with many serious mispronunciations; 0–2 unintelligible or no voice
Sentence	Fluency	8–10 coherent; 6–7 generally coherent with a few pauses/repetition/stammering; 4–5 incoherent with many; 0–3 not able to read as a whole or no voice
Sentence	Prosody	9–10 correct intonation and stable rhythm; 7–8 nearly correct intonation at stable speed; 3–6 unstable speed or inappropriate intonation; 0–2 too stammering or no voice
Sentence	Completeness	Percentage of target words actually pronounced

The original paper states that sentence-level scores vary across 3 to 10, while most word-level and phoneme-level scores fall in the range 8 to 10 when phoneme scores are linearly mapped to a 0–10 plot for visualization. It does not report an aggregation rule across the five raters, nor inter-annotator agreement metrics such as kappa or ICC (Zhang et al., 2021).

Later benchmark work has exposed an important property of the label distribution. Completeness is repeatedly described as ambiguous and highly skewed: one study reports that human completeness ratings are strongly biased toward the maximum score of 10, even when content diverges, while another reports that only 8 training utterances have completeness scores in the range 5–8 and only 14 test utterances have completeness scores between 0–8. This makes completeness unusually difficult for correlation-based evaluation (Parikh et al., 20 Jan 2026, Wang et al., 19 Sep 2025).

A further convention emerged in later work: word- and sentence-level labels are often rescaled from 0–10 to 0–2 for joint modeling with phoneme accuracy, or conversely phoneme labels are linearly mapped from 0–2 to 0–10 for uniform prompting and reporting. Benchmark papers also frequently add “word total” and “utterance total” as supervised targets, even though the original manual-scoring table emphasizes the component aspects rather than a separate total rubric (Gong et al., 2022, Wang et al., 19 Sep 2025).

3. Availability, official baseline, and reproducibility infrastructure

Speechocean762 is distributed through OpenSLR as ID 101 and is explicitly allowed to be used freely for commercial and non-commercial purposes. The original release also provides an open-source Kaldi baseline recipe at egs/gop_speechocean762, making the corpus unusually reproducible by CAPT standards (Zhang et al., 2021).

The official baseline demonstrates a classical GOP workflow. A native acoustic model trained on LibriSpeech is used to generate frame-level posteriors, forced alignment is performed against the expert-voted canonical phone sequences, GOP values and GOP-based features are computed, and per-phone regressors are trained to predict phoneme scores. The primary regressor is an SVR, with polynomial regression on raw GOP values offered as a lighter alternative (Zhang et al., 2021).

A technically important design choice in the Kaldi recipe is direct construction of the LG FST from the expert-selected canonical phone sequences. This avoids the mismatch that would arise if alignment were built from a lexicon containing multiple pronunciations while the manual labels were tied to a single expert-voted sequence. The recipe also addresses score imbalance by supplementing each phone regressor with high-score samples from other phones, randomly relabeled as zero for the current phone (Zhang et al., 2021).

The original baseline reports modest but serviceable phoneme-level performance: using the GOP value directly yields MSE $=0.69$ and PCC $=0.25$, while using GOP-based features improves this to MSE $=0.16$ and PCC $=0.45$ (Zhang et al., 2021).

Later work standardized the official split even further. One influential benchmark paper reports that the training set contains 2,500 utterances, 15,849 words, and 47,076 phones, while the test set contains 2,500 utterances, 15,967 words, and 47,369 phones. This helped establish a consistent public protocol for cross-paper comparison (Gong et al., 2022).

4. Benchmark roles across APA, fluency assessment, MDD, and multimodal scoring

Speechocean762 supports several distinct but overlapping research programs. In multi-aspect, multi-granularity APA, GOPT treats the corpus as a joint regression problem over phoneme accuracy, word accuracy/stress/total, and utterance accuracy/fluency/completeness/prosody/total using GOP features from a public Librispeech acoustic model (Gong et al., 2022). Subsequent systems changed either the feature stack or the inductive bias: 3M augments GOP with duration, energy, and SSL features plus a vowel/consonant positional embedding (Chao et al., 2022); HiPAMA introduces sup-phonemes, depth-wise separable convolution, and score-restraint attention pooling (Chao et al., 2023); Acoustic Feature Mixup explicitly targets score imbalance, especially for Stress and Completeness (Do et al., 2024); and JCAPT combines multi-granular APA with phone-level MDD in a Mamba-based joint framework with phonological features and think tokens (Yang et al., 24 Jun 2025).

The corpus is also a principal benchmark for fluency-specific work. A phonetic and prosody-aware SSL approach uses phone identities, durations, and forced-alignment-derived deep features, then fine-tunes on utterance-level fluency scores (Fu et al., 2023). CBF-AFA reformulates fluency into a three-class problem—Low_fluency $0\!-\!5$, Medium_fluency $6\!-\!7$, and High_fluency $8\!-\!10$—and applies Silero-VAD, breath-group chunking, multi-SSL fusion, and a CNN-BiLSTM over chunk sequences (Wade et al., 25 Jun 2025).

Zero-shot and resource-light assessment lines use Speechocean762 to test how much pronunciation information can be extracted without score supervision. A HuBERT-based masked token recovery method performs sentence-level APA without ASR, text references, or fine-tuning (Liu et al., 2023). A multimodal LLM system based on Data2Vec2, an m-adapter, and a frozen Qwen-7B predicts sentence-level accuracy and fluency from audio plus prompt text (Fu et al., 2024). A later lightweight framework discretizes speech with HuBERT and K-means, models native token surprisal with a 3-gram LM, and adds transcript-guided Text2DUnit--DTW alignment features trained only on native speech resources (Sara et al., 18 Jun 2026).

SpeechLLM and LMM studies have further expanded the benchmark’s role. Qwen2-Audio-7B-Instruct has been evaluated zero-shot for sentence-level accuracy, fluency, prosody, and completeness (Parikh et al., 20 Jan 2026). GPT-4o has been used in an align-free zero-shot setup for multi-level scoring and feedback generation (Wang et al., 14 Mar 2025). Fine-tuned multimodal systems include a LoRA-adapted Phi-4 model for joint APA and MDD (Ahn et al., 3 Sep 2025), a fine-tuned Qwen2-Audio model with SimPO or DPO-style preference learning for multi-granular APA (Wang et al., 19 Sep 2025), and a rubric-guided SpeechLLM that jointly predicts sentence-, word-, and phoneme-level labels while generating natural-language rationales (Parikh et al., 8 Jun 2026).

MDD and phone-level pronunciation scoring remain equally central. CoCA-MDD uses coupled cross-attention between acoustic and reference-phone streams and reports phone-level scoring on Speechocean762 (Zheng et al., 2021). SpeechBlender augments phoneme-level training data by smoothly blending donor and candidate phone segments selected from confusion pairs (Kheir et al., 2022). L1-MultiMDD treats Speechocean762 as an unseen English-by-Mandarin test set for multilingual zero-shot generalization, including both adults and children (Kheir et al., 2023). Substitution-aware alignment-free GOP constrains phoneme substitutions through a handcrafted phoneme confusion map and evaluates all 5,000 utterances without relying on forced alignment in the scoring path (Parikh et al., 2 Jun 2025).

5. Representative empirical results

Reported numbers on Speechocean762 are highly task-dependent and are not directly comparable across phoneme regression, sentence-level APA, three-class fluency classification, and MDD. Even so, a few results have become reference points for the benchmark:

Setting	Representative reported result	Source
Original Kaldi baseline	Phoneme-level PCC $\le 30\%$0, MSE $\le 30\%$1 with GOP-based features	(Zhang et al., 2021)
GOPT	Phoneme PCC $\le 30\%$2; utterance total PCC $\le 30\%$3	(Gong et al., 2022)
Prosody-aware SSL fluency scoring	Best fluency PCC $\le 30\%$4 with BLSTM-pre and duration-only reconstruction	(Fu et al., 2023)
Zero-shot HuBERT APA	Sentence-level PCC $\le 30\%$5; supervised GoP baseline $\le 30\%$6; GOPT $\le 30\%$7	(Liu et al., 2023)
Chunk-based fluency classification	F1 $\le 30\%$8, PCC $\le 30\%$9	(Wade et al., 25 Jun 2025)
Joint APA+MDD with JCAPT	Phoneme PCC $>30\%$0; utterance total PCC $>30\%$1; MDD F1 $>30\%$2	(Yang et al., 24 Jun 2025)
Native-resource-only discrete token surprisal with transcript guidance	Accuracy/Fluency/Prosody PCC $>30\%$3	(Sara et al., 18 Jun 2026)

Zero-shot speech-LLM behavior on the corpus is especially revealing. Qwen2-Audio-7B-Instruct achieves within-$>30\%$4 agreement of $>30\%$5, $>30\%$6, and $>30\%$7 for accuracy, fluency, and prosody, but the raw PCCs are only $>30\%$8, $>30\%$9, and $=0.69$0, while completeness drops to $=0.69$1. The same study reports a systematic overestimation bias for low-quality speech, including $=0.69$2 low-score coverage for utterances with ground-truth accuracy $=0.69$3 (Parikh et al., 20 Jan 2026).

Fine-tuned multimodal models narrow that gap but do not remove it uniformly across granularities. A LoRA-tuned Phi-4 multimodal model trained on Speechocean762 reports sentence-level PCCs up to $=0.69$4 for accuracy, $=0.69$5 for fluency, and $=0.69$6 for prosodic, while achieving WER $=0.69$7 and PER $=0.69$8 in joint APA/MDD evaluation (Ahn et al., 3 Sep 2025). A fine-tuned Qwen2-Audio model reaches competitive word- and sentence-level results, but its phoneme-level PCC remains $=0.69$9 in multi-granularity experiments, below specialized baselines such as GOPT and HMamba (Wang et al., 19 Sep 2025).

At the phone level, MDD-oriented methods also show the corpus’s diagnostic range. CoCA-MDD reports phone-level PCC $=0.25$0, improving over a GOP-NN + SVR baseline at $=0.25$1 (Zheng et al., 2021). SpeechBlender raises phoneme-level PCC to $=0.25$2 with MSE $=0.25$3 through augmentation (Kheir et al., 2022). In alignment-free GOP, the PP-AF GOP UPS variant reports AUC $=0.25$4, MCC $=0.25$5, and phoneme-score PCC $=0.25$6 under the “high conf” setting (Parikh et al., 2 Jun 2025).

6. Strengths, caveats, and continuing significance

Speechocean762 has several properties that explain its longevity. It is open-source and free for both commercial and non-commercial use; it includes both children and adults; it provides sentence-, word-, and phoneme-level annotations from five experts; it stores expert-voted canonical phone sequences; and it ships with a public Kaldi baseline recipe (Zhang et al., 2021). A plausible implication is that few public CAPT corpora offer as clean a bridge between classical forced-alignment pipelines, SSL-based assessment, zero-shot methods, and modern multimodal generative models.

At the same time, several limitations recur throughout the literature. The corpus is Mandarin-L1 only, so cross-L1 generalization must be established elsewhere or through external zero-shot testing (Zhang et al., 2021, Kheir et al., 2023). It is also a read-aloud benchmark with known prompts, which strongly favors text-conditioned assessment and may underrepresent open-response or spontaneous-speech phenomena (Zhang et al., 2021, Fu et al., 2024). The original release does not report the aggregation rule across five raters, inter-annotator agreement metrics, or detailed audio specifications such as sampling rate and bit depth (Zhang et al., 2021).

Label imbalance is a second persistent issue. Later studies repeatedly identify severe skew in Completeness and, to a lesser degree, Stress or low-score regions more generally (Parikh et al., 20 Jan 2026, Do et al., 2024, Wang et al., 19 Sep 2025). This suggests that reported gains on those aspects should be interpreted together with the task formulation, the metric choice, and the score transformation used in each paper. The same caveat applies to phoneme labels: the native phoneme scale is 0–2, but many later systems rescale it to 0–10 for unified prompting or joint training, which changes the numerical regime of evaluation without altering the underlying annotation granularity (Wang et al., 19 Sep 2025).

A further pattern is selective use of the corpus. Many studies exploit only sentence-level labels, only fluency, or only phoneme accuracy, even though the dataset was designed as a multi-granularity benchmark (Fu et al., 2023, Liu et al., 2023). This reflects both the richness of the corpus and the engineering difficulty of using all annotation levels simultaneously.

Despite these caveats, Speechocean762 remains active well into 2026 as a sole or primary public benchmark for joint multi-granular assessment, speech-LLM rationales, and native-resource-only pronunciation scoring (Parikh et al., 8 Jun 2026, Sara et al., 18 Jun 2026). Its continuing relevance lies in a rare combination: public access, controlled prompts, expert multi-level labels, and a benchmark history broad enough to connect GOP-era CAPT, SSL-based assessment, alignment-free methods, and instruction-following multimodal models within a single shared evaluation setting.