Pronunciation Error Rate (PrER)

• Pronunciation Error Rate (PrER) is an evaluation metric that quantifies pronunciation accuracy by comparing predicted articulatory attribute sequences with reference sequences.
• It differentiates between exact-match string errors and edit-distance based errors, enabling nuanced assessment across phoneme, phone, and articulatory levels.
• Recent studies use PrER alongside related metrics like PER, WPER, and AER to diagnose acoustic model performance and guide improvements in pronunciation assessment.

Pronunciation Error Rate (PrER) denotes an error-rate measure intended to quantify how accurately a system captures pronunciation rather than only orthographic, lexical, or syllabic labels. In the current literature, the term has both a narrow and a broader usage. One recent ASR paper introduces PrER explicitly as an articulatory-attribute-sequence error rate for syllable-based recognition (Yen et al., 9 Sep 2025). Several related works do not name PrER, but evaluate pronunciation with exact-match pronunciation error, Phone Error Rate (PER), Phoneme Error Rate (PER), Weighted Phoneme Error Rate (WPER), Articulatory Error Rate (AER), or ASR-derived mismatch features such as CER and MER (Jones et al., 2022, Hosseini-Kivanani et al., 2021, Zhou et al., 18 Jul 2025, Do et al., 2024, Abdelfattah et al., 27 Aug 2025). This suggests that PrER is best understood not as a single universally standardized metric label, but as a pronunciation-sensitive family of mismatch measures whose meaning depends on the representation being compared.

1. Terminological status across the literature

The most explicit use of the term appears in "A Bottom-up Framework with Language-universal Speech Attribute Modeling for Syllable-based ASR" (Yen et al., 9 Sep 2025). There, PrER is introduced to evaluate how accurately a model captures the underlying pronunciation, independent of orthographic symbols, by converting outputs into attribute sequences and comparing them with reference attribute sequences. The paper treats PrER as an articulatory-level diagnostic for acoustic modeling rather than as a syllable-level recognition score.

By contrast, several pronunciation-assessment papers use adjacent metrics without adopting the label PrER. The Japanese geographic-name study reports exact-match “error rate” and “accuracy” on predicted hiragana pronunciations, with error rate equal to the proportion of items whose predicted pronunciation differs from the reference (Jones et al., 2022). The mispronunciation-detection study for Italian learners of English uses phone-level PER as its principal pronunciation-related metric (Hosseini-Kivanani et al., 2021). The phonetic error detection paper proposes WPER and AER in addition to standard PER, and identifies WPER as the pronunciation-sensitive metric most comparable to a PrER-like quantity (Zhou et al., 18 Jul 2025). The multi-aspect pronunciation-assessment work uses ASR-derived CER and MER as auxiliary pronunciation-error proxies rather than a metric named PrER (Do et al., 2024). The Quran recitation work reports PER across a phoneme level and ten sifat levels, but does not define PrER (Abdelfattah et al., 27 Aug 2025).

Paper	Reported metric	Relation to PrER
(Yen et al., 9 Sep 2025)	PrER	Explicit attribute-sequence pronunciation error rate
(Jones et al., 2022)	Error rate / accuracy	Exact-match pronunciation mismatch rate
(Hosseini-Kivanani et al., 2021)	PER	Phone-level pronunciation error proxy
(Zhou et al., 18 Jul 2025)	PER, WPER, AER	WPER is the closest PrER-like metric
(Do et al., 2024)	CER, MER	ASR-derived pronunciation-error proxies
(Abdelfattah et al., 27 Aug 2025)	PER	Multi-level phoneme/sifa error rate without the PrER label

2. Units of comparison and operational definitions

The central distinction among PrER-like measures is the representational unit on which error is computed. In the geographic-name task, the unit is the full pronunciation string: the input is a Japanese geographic feature name in kanji/kana, and the output is its pronunciation in hiragana. Evaluation is exact match against the database pronunciation, so any mismatch in the predicted hiragana string counts as an error. In that setting, the effective pronunciation error rate is simply $1 - \text{accuracy}$ under exact-match evaluation (Jones et al., 2022).

In phone- and phoneme-based mispronunciation work, the unit is a phone or phoneme sequence rather than a full lexical pronunciation string. The English learner study defines PER by substitutions, deletions, and insertions relative to the number of phones in the reference transcript, and uses it as the primary measure for assessing phoneme performance (Hosseini-Kivanani et al., 2021). The phonetic error detection framework likewise defines PER at the phoneme-sequence level and uses it to score recognition outputs against actual spoken phoneme sequences rather than intended canonical forms (Zhou et al., 18 Jul 2025).

Other works move beyond canonical phoneme sequences. The Quran recitation system introduces a Quran Phonetic Script (QPS) with two levels—phoneme level and sifa level—and predicts 11 parallel labels for each utterance: one phoneme-level transcript and ten sifat-level outputs. It then reports per-level PER values such as per_phonemes, per_ghonna, and average_per (Abdelfattah et al., 27 Aug 2025). The bottom-up syllable ASR framework defines PrER over articulatory attribute sequences such as Manner, Place, Voicing, Aspiration, Height, and Backness, and permits computation on a single attribute category or on combinations of attributes (Yen et al., 9 Sep 2025). The multi-aspect pronunciation-assessment paper uses ASR output aligned to reference phonemes to derive CER and MER, which are then concatenated with learned acoustic representations (Do et al., 2024).

These choices are not merely notational. They determine what kind of pronunciation deviation is visible to the metric. Exact-match string error collapses all deviations into a binary outcome. Phone- or phoneme-level PER exposes edit operations. Attribute-level PrER separates articulatory dimensions that may be obscured by syllable or orthographic labels. Multi-level phonetic encodings such as QPS further decompose pronunciation into phoneme identity and articulation-related properties.

3. Sequence-error formulations and weighted variants

The simplest PrER-like regime in the surveyed literature is exact-match pronunciation error. In the Japanese geographic-name study, a case is counted as an error whenever the predicted pronunciation does not exactly match the reference pronunciation in the maps database. This yields a pronunciation mismatch rate that is equivalent to $1 - \text{accuracy}$ (Jones et al., 2022). The same binary logic underlies many lexicon-oriented pronunciation prediction tasks, but it does not distinguish between a minor deviation and a severe mismatch.

PER refines this by using edit operations. In the ASR-based mispronunciation-detection system for Italian learners of English, PER is defined in terms of substitutions, deletions, and insertions relative to the number of phones in the reference transcript, and functions as the main pronunciation error proxy (Hosseini-Kivanani et al., 2021). Conceptually, this aligns pronunciation assessment with standard sequence-recognition error accounting.

A more explicitly pronunciation-sensitive variant is WPER, introduced in "Towards Accurate Phonetic Error Detection Through Phoneme Similarity Modeling" (Zhou et al., 18 Jul 2025). Its formula is

$$WPER = \frac{D + \displaystyle \sum^{(p_r, p_s)} (1 - S(p_r, p_s)) + I}{L}$$

where $L$ is the length of the reference phoneme sequence, $D$ is the number of deletions, $I$ is the number of insertions, $p_r$ and $p_s$ are the reference and substituted phonemes, and $S$ is the normalized phoneme similarity matrix. A substitution penalty is therefore $1 - S(p_r,p_s)$, so substitutions between similar phonemes contribute less than substitutions between dissimilar phonemes. The same paper also introduces AER, defined by converting speech to articulatory features using an acoustic-to-articulatory inversion model, labeling frames as negative when their $1 - \text{accuracy}$0 distance from the target phoneme representation exceeds a threshold $1 - \text{accuracy}$1, and computing the ratio of negative frames to total speech length (Zhou et al., 18 Jul 2025).

The explicit PrER of the bottom-up syllable ASR framework is operational rather than formulaic. Model outputs are first converted into attribute sequences using a syllable-to-attribute mapping, after which predicted and reference attribute sequences are compared and the resulting error rate is reported as PrER (Yen et al., 9 Sep 2025). The paper does not print a closed-form equation, but the procedure is sequence-based and is analogous in spirit to recognition error rates computed over an articulatory representation rather than over phones or syllables.

4. Diagnostic role in pronunciation modeling

PrER has a distinctive diagnostic role in systems whose architecture separates acoustic modeling from lexical or orthographic realization. The bottom-up syllable ASR framework uses PrER specifically to test whether the acoustic model has learned the underlying pronunciation structure. Because syllable outputs can hide pronunciation detail, the paper evaluates PrER without a LLM during decoding in order to isolate pure acoustic modeling performance (Yen et al., 9 Sep 2025). This makes PrER a diagnostic metric rather than merely a task-level success criterion.

That paper introduces PrER alongside two syllable-level metrics: SER and SHER. SER is the standard Syllable Error Rate. SHER is a relaxed metric that treats syllables sharing the same underlying attribute sequence as equivalent, thereby accounting for homonym ambiguity in the attribute-to-syllable mapping. The intended interpretation is explicit: if SER is high but PrER is low, the system has learned pronunciation well but remains under-informed in the mapping to syllables; if SHER is substantially lower than SER, many nominal syllable errors are compatible with the correct underlying pronunciation (Yen et al., 9 Sep 2025).

A structurally similar decomposition appears in the Quran recitation work, even though it does not use the term PrER. There, pronunciation error detection is broadened beyond phoneme identity to include articulation errors, attribute errors, and Tajweed rule errors, using one phoneme-level output head and ten sifat-level heads in a multi-level CTC architecture (Abdelfattah et al., 27 Aug 2025). This is not labeled PrER, but it serves a comparable analytic purpose: it localizes pronunciation deviations to subcomponents of the phonetic representation.

The broader literature therefore suggests that PrER-like metrics become most informative when the output representation itself is linguistically structured. In such settings, a single scalar error rate is less important than the representational layer at which the error is measured.

5. Empirical behavior across tasks

In Japanese geographic-feature pronunciation prediction, neighbor information materially reduces exact-match pronunciation error. On the shuffled test set, the transformer without neighbors reports an error rate of $1 - \text{accuracy}$2, whereas the transformer with $1 - \text{accuracy}$3 neighbors reports $1 - \text{accuracy}$4. On the geographically disjoint unshuffled test set, the same comparison is $1 - \text{accuracy}$5 versus $1 - \text{accuracy}$6. The paper states that confidence intervals do not overlap, paired bootstrap resampling supports significance, and a paired permutation test gives $1 - \text{accuracy}$7 (Jones et al., 2022). For a 60-item sample of errors from the best model, $1 - \text{accuracy}$8 were judged “recoverable,” indicating that exact-match error can overstate the practical severity of some pronunciation mismatches (Jones et al., 2022).

In phonetic error detection, similarity-aware scoring and training substantially alter measured pronunciation error. On VCTK-accent, adding both soft phoneme mapping and soft CTC reduces PER from $1 - \text{accuracy}$9 to $$WPER = \frac{D + \displaystyle \sum^{(p_r, p_s)} (1 - S(p_r, p_s)) + I}{L}$$0, WPER from $$WPER = \frac{D + \displaystyle \sum^{(p_r, p_s)} (1 - S(p_r, p_s)) + I}{L}$$1 to $$WPER = \frac{D + \displaystyle \sum^{(p_r, p_s)} (1 - S(p_r, p_s)) + I}{L}$$2, and AER from $$WPER = \frac{D + \displaystyle \sum^{(p_r, p_s)} (1 - S(p_r, p_s)) + I}{L}$$3 to $$WPER = \frac{D + \displaystyle \sum^{(p_r, p_s)} (1 - S(p_r, p_s)) + I}{L}$$4. Among the similarity models, the articulatory-based similarity performs best on VCTK-accent for WPER and AER (Zhou et al., 18 Jul 2025). These results show that the weighting scheme embedded in WPER is not a cosmetic variant of PER; it changes the effective sensitivity of the evaluation to phonetic closeness.

In ASR-based mispronunciation detection for Italian learners of English, the reported n-gram systems yield PER values generally in the range $$WPER = \frac{D + \displaystyle \sum^{(p_r, p_s)} (1 - S(p_r, p_s)) + I}{L}$$5 to $$WPER = \frac{D + \displaystyle \sum^{(p_r, p_s)} (1 - S(p_r, p_s)) + I}{L}$$6. For the 5-gram model, PER is $$WPER = \frac{D + \displaystyle \sum^{(p_r, p_s)} (1 - S(p_r, p_s)) + I}{L}$$7 for ISLE and $$WPER = \frac{D + \displaystyle \sum^{(p_r, p_s)} (1 - S(p_r, p_s)) + I}{L}$$8 for ChildEn in the ASR-versus-MAN comparison. The adapted error LLM achieves the lowest reported PER, $$WPER = \frac{D + \displaystyle \sum^{(p_r, p_s)} (1 - S(p_r, p_s)) + I}{L}$$9, and ChildEn detection accuracy is reported around $L$0–$L$1 for correct phone-error detection with the error model (Hosseini-Kivanani et al., 2021). Here PER functions as a pronunciation error proxy in a CAPT-style setting.

The Quran recitation system reports exceptionally low recognition error on clean test data: average_per = 0.0016, interpreted as $L$2, with per_phonemes = 0.00543 and similarly low per-sifa values such as per_qalqla = 0.00085 and per_ghonna = 0.0013 (Abdelfattah et al., 27 Aug 2025). These figures are reported on unseen Mushafs and demonstrate learnability of the QPS representation under a multi-level CTC model, rather than conventional learner mispronunciation detection.

For explicit PrER, the bottom-up syllable ASR framework reports attribute-specific values on AISHELL-1 without language-model decoding. In the full-data setting, $L$3 yields M, P, H, B, A scores of $L$4, whereas $L$5 yields $L$6, improving every reported attribute category (Yen et al., 9 Sep 2025). In zero-shot Japanese evaluation on JSUT, PrER is much higher, with ZH $L$7 JP giving M, P, V, H, B of $L$8 and MLS $L$9 JP giving $D$0 (Yen et al., 9 Sep 2025). The paper uses these results to argue that the universal attribute inventory transfers across languages, while also showing the difficulty of zero-shot pronunciation modeling.

6. Limitations, caveats, and recurring misconceptions

A recurrent misconception is to treat PrER as a fixed, universally agreed metric name. The surveyed literature does not support that assumption. Several papers central to pronunciation assessment do not define PrER explicitly and instead report PER, exact-match error rate, WPER, AER, CER, or MER (Jones et al., 2022, Hosseini-Kivanani et al., 2021, Zhou et al., 18 Jul 2025, Do et al., 2024, Abdelfattah et al., 27 Aug 2025). Even when the underlying intent is pronunciation-sensitive evaluation, the metric label and scoring unit vary substantially.

Another important caveat is label reliability. The Japanese geographic-name paper emphasizes that the maps database contains some wrong pronunciations, so a model prediction can be marked “wrong” by the metric even when the model is actually correct (Jones et al., 2022). Exact-match pronunciation error therefore conflates model error with annotation error. A related but distinct issue appears in the multi-aspect pronunciation-assessment work: CER and MER are extracted from ASR hypotheses, so ASR errors can propagate into the downstream pronunciation model. The paper explicitly notes that excluding error-rate features in static and dynamic mixups yields better performance for some severely imbalanced aspects (Do et al., 2024).

Dataset construction also affects interpretation. VCTK-accent is a TTS-generated dataset with simulated phonetic errors, though the phonetic error detection paper additionally evaluates on L2-ARCTIC, Speechocean762, MultiPA, and PPA Speech with manually labeled actual pronunciations (Zhou et al., 18 Jul 2025). The Quran recitation paper states that its dataset consists of golden recitations with no errors and therefore cannot yet support comprehensive evaluation on real learner mispronunciations; the reported PER reflects sequence recognition performance on clean data, not end-to-end pronunciation assessment under ordinary learner-speech protocols (Abdelfattah et al., 27 Aug 2025).

Finally, some formulations remain operational rather than formal. The explicit PrER paper describes the computation procedure in prose rather than giving a standalone equation (Yen et al., 9 Sep 2025). The Quran recitation work likewise reports PER values without an explicit formula (Abdelfattah et al., 27 Aug 2025). This does not make the metrics unusable, but it complicates direct comparison across papers and reinforces the need to inspect the representational level and alignment procedure behind any reported pronunciation error rate.