ReNikud: Audio-Supervised Hebrew Grapheme-to-Phoneme Conversion

Abstract: Grapheme-to-phoneme (G2P) conversion for Modern Hebrew is needed for applications like text-to-speech (TTS), but is challenging due to the language's abjad writing system, which leaves vowels largely unwritten, creating substantial ambiguity. Standard approaches first predict vowel diacritics (nikud) to produce International Phonetic Alphabet (IPA) transcriptions, but this is limited: vocalization data is scarce and laborious to produce, it does not specify features such as lexical stress, and it reflects formal grammatical rules rather than everyday spoken pronunciation. Direct sequence-to-sequence IPA prediction, meanwhile, struggles on limited data and fails to exploit the character-level alignment characteristic of abjads. Our method, ReNikud, overcomes these limitations with two key insights: (1) Weak audio supervision via a phoneme-based automatic speech recognition (ASR) pseudo-labeling pipeline on thousands of hours of unlabeled Hebrew audio, yielding phonemic transcriptions that reflect natural spoken norms without manual annotation. (2) A pseudo-vocalization architecture that predicts IPA phonemes at each character position, enforcing character-level alignment as an inductive bias. Results on existing Hebrew G2P benchmarks and the new targeted MILIM benchmark for spoken Hebrew show that ReNikud surpasses previous state-of-the-art methods. We will release our code and trained models to support further work on Hebrew TTS and speech technologies.

• The paper introduces an audio-supervised training pipeline that extracts character-aligned IPA labels from thousands of hours of unlabeled Hebrew audio using dual ASR outputs.
• It presents a pseudo-vocalization architecture with a character-level Transformer and three dedicated classification heads for predicting consonant, vowel, and stress, ensuring legal phoneme realizations.
• Experimental results show significant improvements on the MILIM benchmark with 22.8% WER and 6.5% CER, validating the method in colloquial speech and low-resource diacritization tasks.

ReNikud: Audio-Supervised Hebrew Grapheme-to-Phoneme Conversion

Problem Formulation and Motivation

Hebrew G2P conversion presents unique challenges due to its abjad orthography, in which vowels are largely unwritten, resulting in high ambiguity. Existing TTS systems rely on intermediate diacritization, but classical diacritics fail to encode crucial phonetic features such as lexical stress and diverge from contemporary spoken pronunciation. Manual vocalization and IPA annotation data are scarce and costly, limiting the scalability of text-based approaches. Direct sequence-to-sequence IPA prediction struggles with limited resources and fails to leverage the character-level alignment inherent to abjads. There exists a substantial corpus of untranscribed Hebrew audio reflecting natural spoken pronunciation, yet previous G2P solutions are unable to exploit this data source.

Methodological Contributions

Audio-Based Weak Supervision

ReNikud introduces an innovative training pipeline extracting phonemic supervision from thousands of hours of unlabeled Hebrew audio. The process utilizes parallel ASR systems: one produces Hebrew orthographic transcripts, while the other is custom-fine-tuned to output IPA transcriptions. A monotonic finite state transducer (FST) aligns Hebrew graphemes and IPA phonemes, enforcing character-level correspondence. This pipeline creates large-scale, character-aligned IPA pseudo-labels, reflecting actual spoken norms without manual annotation.

Quality controls retain only samples with successful alignment and matching word counts between ASR outputs. The pipeline fragments long utterances at alignment failures, substantially increasing the sample count while filtering out ASR noise and disfluencies.

Pseudo-Vocalization Architecture

The G2P model adopts a character-level Transformer encoder. Three independent parallel classification heads predict, for each grapheme, the corresponding IPA consonant, vowel, and stress indicator. This explicit per-character mapping serves as an inductive bias, leveraging the abjad structure for increased efficiency and accuracy. Constrained decoding restricts predictions to legal phoneme realizations per grapheme, and enforces exactly one lexical stress per word, significantly improving WER and CER compared to unconstrained outputs.

MILIM Benchmark

The MILIM benchmark systematically evaluates the effectiveness of G2P models on diverse categories of spoken Hebrew: slang, loanwords, acronyms, rare phonemes, colloquialisms, gender-specific forms, homographs, stress pairs, and proper names. Each category contains semi-manual IPA annotations paired with sentences, focusing on items that exhibit the largest divergence between written and spoken norms.

Experimental Results

G2P Performance

On MILIM, ReNikud achieves marked performance improvements, especially on items reflecting modern spoken Hebrew. The model yields strong numerical gains: 22.8% WER and 6.5% CER overall, outperforming prior baselines (Phonikud: 32.7% WER, 10.0% CER). Gains are most pronounced in colloquial, slang, foreign, and penultimate stress categories with absolute improvements exceeding 10% WER. Qualitative comparisons reveal superior handling of stress placement, rare phonemes, and informal spoken forms, where text-supervised baselines falter.

In comparison to LLM-based approaches (Gemini 3.1 Pro, GPT-5.5 High, Opus 4.8 High Thinking), ReNikud targets practical deployment scenarios, offering reproducibility, low latency, and open accessibility, unlike proprietary LLMs, which though competitive in accuracy, remain impractical for large-scale or real-time Hebrew TTS.

User Study Validation

A native speaker preference study validates MILIM's annotations as aligning with colloquial spoken norms. Listeners consistently preferred informal variants synthesized via ReNikud, achieving a mean CMOS of +0.87, statistically significant ($p < 0.001$) in favor of the MILIM pronunciation inventory.

Transfer Learning for Diacritization

ReNikud's audio-supervised encoder shows transferability to traditional diacritization tasks. When fine-tuned with minimal labeled data (down to 1,000 sentences), the model achieves faster convergence and lower WER compared to DictaBERT (23.6% vs. 37.4% WER at 1k; 13.4% vs. 15.7% at 10k), demonstrating the utility of audio-derived phonetic pretraining in extreme low-resource scenarios.

Ablation Studies

Ablations confirm the advantages of the pseudo-vocalization architecture: character-level heads outperform seq2seq and CTC baselines in both stress and vowel accuracy. Audio-derived IPA supervision yields lower error rates compared to text-derived IPA, particularly in spoken categories where written and spoken norms diverge.

Related Work and Distinctions

Prior Hebrew G2P works (Kolani et al., Phonikud; DictaBERT; Nakdimon) depend on costly labeled text and fail to capture spoken language variation. Recent English and multilingual G2P studies using supervised audio rely on labeled text-audio pairs, whereas ReNikud leverages unlabeled audio as primary supervision, a distinctive feature fitting abjad languages with high orthographic ambiguity. Audio-guided inference methods, which require speech input at inference, are unsuitable for text-only applications such as TTS.

Implications and Future Directions

Practically, ReNikud enables scalable, accurate G2P conversion for Hebrew TTS compatible with spoken norms, particularly benefitting applications demanding precise phonetic transcriptions (voice assistants, speech-driven UI, and generative speech modeling). The pipeline's reliance on unlabeled audio democratizes G2P learning, bypassing data bottlenecks associated with manual annotation.

Theoretically, the explicit induction of character-to-phoneme mapping aligned with abjad structure underpins improved generalization and data efficiency, and establishes a precedent for exploiting structural biases in low-resource orthographies.

Future research may target the following objectives:

• Incorporation of conversational and diverse audio sources to remedy formal-domain bias of current corpora.
• Adaptation of the pipeline for Arabic and other abjad languages, exploiting shared properties of unwritten vowels and spoken/written divergence.
• Refinements in ASR accuracy and FST alignment to handle extreme colloquial variances.
• Extension to multilingual benchmarks integrating both abjad and non-abjad languages for comparative evaluation.

Conclusion

ReNikud presents a robust audio-supervised G2P system for Hebrew, delivering strong numerical gains and accurate modeling of spoken language phenomena through a character-aligned pseudo-vocalization architecture. Its methodology unlocks large-scale pronunciation learning from unlabeled audio, sets new standards for evaluating spoken Hebrew G2P via MILIM, and demonstrates benefits for related tasks such as diacritization. The framework's architecture and training strategies are poised for adaptation to other abjad languages and broader speech technology domains (2606.20179).