LatPhon: Compact Multilingual G2P

• LatPhon is a compact multilingual G2P model that converts Latin-script graphemes to IPA phonemes, serving as a unified front-end for speech systems.
• It uses a small Transformer with 4 encoder-decoder layers and rotary positional encodings to achieve low phoneme error rates across six languages.
• The model’s 30MB footprint and efficient inference enable practical on-device deployment for TTS, ASR, S2ST, and forced alignment pipelines.

LatPhon is a compact, fully autoregressive grapheme-to-phoneme (G2P) converter based on a small Transformer, jointly trained on six Latin-script languages—English, Spanish, French, Italian, Portuguese, and Romanian. It is positioned as a single multilingual front-end for text-to-speech (TTS), automatic speech recognition (ASR), speech-to-speech translation (S2ST), and alignment systems, with an explicit design trade-off among model size, multilingual coverage, and inference speed. On the public ipa-dict corpus, it attains a mean phoneme error rate (PER) of 3.5%, outperforming a 580 M-parameter ByT5 baseline at 5.4% and approaching per-language WFSTs at 3.2%, while occupying 30 MB of memory, which makes on-device deployment feasible when needed (Chary et al., 3 Sep 2025).

1. Definition and problem setting

LatPhon addresses multilingual G2P conversion for Latin-script languages. In this setting, the input is a grapheme sequence and the output is an IPA phoneme sequence. The model is intended to replace language-specific front-ends with a shared multilingual component, while retaining accuracy close to stronger specialized systems. The underlying motivation is operational: G2P is a key front-end for TTS, ASR, S2ST, and forced alignment, and a single deployable model reduces system fragmentation across speech pipelines (Chary et al., 3 Sep 2025).

The system’s scope is deliberately constrained. It is trained on dictionary-style (word, IPA) pairs from ipa-dict rather than on sentence-level contextual phonemization. Accordingly, its primary function is lexical pronunciation prediction rather than higher-level disambiguation of context-sensitive forms. This suggests that LatPhon should be understood as a compact multilingual lexical front-end rather than as a full sentence-aware phonological analyzer.

A central design claim is that compact multilingual G2P can serve as a universal front-end for Latin-language speech pipelines. In context, this means that the model is not optimized solely for best-in-class monolingual accuracy; it is optimized for a deployment regime in which one small model can cover multiple languages with real-time inference and modest memory requirements.

2. Architecture and parameterization

LatPhon adopts a standard encoder-decoder Transformer, scaled down to four layers in both encoder and decoder, with hidden dimension $d_{\mathrm{model}} = 256$ and 8 attention heads. Rotary positional encodings replace fixed sinusoidal ones, injecting position information directly into the queries and keys. Grapheme characters are embedded in 256-dimensional vectors, and a learned 6-way language-ID embedding is prepended to the input sequence to condition the model on the target language. The decoder predicts over a 109-symbol IPA inventory through a standard softmax layer (Chary et al., 3 Sep 2025).

The attention mechanism is the standard multi-head form:

$$\mathrm{MultiHead}(Q,K,V) = \mathrm{Concat}(\mathrm{head}_1,\dots,\mathrm{head}_h)\,W^O,$$

with

$$\mathrm{head}_i = \mathrm{Attention}(QW_i^Q,\,K W_i^K,\,V W_i^V),$$

and

$$\mathrm{Attention}(Q,K,V) = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr)\,V.$$

The compactness of the model follows from the parameter budget, shallow depth, and cross-lingual sharing.

Component	Approx. parameters	Role
Embeddings (graphemes + IPA + language ID)	$\approx 1.5$ M	Input/output symbol representations
Encoder & decoder layers	$\approx 6$ M	Self-attention, cross-attention, feed-forward computation
Output softmax projection	$\approx 0.4$ M	Distribution over the 109-symbol IPA inventory

The total parameter count is approximately 7.5 million. The learned language-ID embedding itself is only 1 k parameters. Keeping $d_{\mathrm{model}}=256$, limiting depth to 4 layers, and sharing most parameters across languages are the principal mechanisms by which LatPhon remains compact. This suggests that multilinguality is treated as a parameter-sharing problem rather than as an ensemble of loosely coupled monolingual models.

3. Training corpus and joint multilingual learning

LatPhon is trained on the ipa-dict corpus, which provides (word, IPA) pairs cleaned of unsupported diacritics. The six languages and train-set sizes are English with 133,969 entries, Spanish with 594,899 entries, French with 245,465 entries, Italian with 6,108 entries, Portuguese with 94,942 entries, and Romanian with 71,375 entries. Validation and test splits each contain 500 words per language (Chary et al., 3 Sep 2025).

Training is jointly multilingual. Examples from all languages are intermixed in each mini-batch of size 64, and a language-ID token indicates the target phoneme set. Optimization uses negative log-likelihood with AdamW, learning rate $3 \times 10^{-4}$, 10 k warm-up steps, cosine decay, and a total of 100 k steps on a single RTX 4090, for approximately 46 minutes. The training objective is

$$\mathcal{L} \;=\; -\sum_{t=1}^T \log P\bigl(p_t \mid p_{<t},\,g,\,\ell\bigr),$$

where $$\mathrm{MultiHead}(Q,K,V) = \mathrm{Concat}(\mathrm{head}_1,\dots,\mathrm{head}_h)\,W^O,$$0 is the grapheme sequence and $$\mathrm{MultiHead}(Q,K,V) = \mathrm{Concat}(\mathrm{head}_1,\dots,\mathrm{head}_h)\,W^O,$$1 is the language ID.

The data distribution is highly imbalanced, with Italian representing a markedly lower-resource condition than Spanish or French. That imbalance is operationally important because the model is jointly trained rather than balanced by separate language-specific optimization. A plausible implication is that LatPhon’s multilingual sharing is most beneficial when enough lexical signal exists for each language to stabilize its pronunciation regularities.

4. Evaluation and comparative performance

Performance is measured by phoneme error rate:

$$\mathrm{MultiHead}(Q,K,V) = \mathrm{Concat}(\mathrm{head}_1,\dots,\mathrm{head}_h)\,W^O,$$2

where $$\mathrm{MultiHead}(Q,K,V) = \mathrm{Concat}(\mathrm{head}_1,\dots,\mathrm{head}_h)\,W^O,$$3, $$\mathrm{MultiHead}(Q,K,V) = \mathrm{Concat}(\mathrm{head}_1,\dots,\mathrm{head}_h)\,W^O,$$4, and $$\mathrm{MultiHead}(Q,K,V) = \mathrm{Concat}(\mathrm{head}_1,\dots,\mathrm{head}_h)\,W^O,$$5 are substitutions, deletions, and insertions, and $$\mathrm{MultiHead}(Q,K,V) = \mathrm{Concat}(\mathrm{head}_1,\dots,\mathrm{head}_h)\,W^O,$$6 is the total number of reference phonemes. On ipa-dict test sets, LatPhon achieves a mean PER of 3.5%, compared with 5.4% for the byte-level ByT5 baseline and 3.2% for per-language WFSTs implemented with Phonetisaurus (Chary et al., 3 Sep 2025).

The per-language results are heterogeneous. English is the most difficult setting among the six, with LatPhon at 12.7% PER $$\mathrm{MultiHead}(Q,K,V) = \mathrm{Concat}(\mathrm{head}_1,\dots,\mathrm{head}_h)\,W^O,$$7, ByT5 at 14.0%, and WFST at 10.4% $$\mathrm{MultiHead}(Q,K,V) = \mathrm{Concat}(\mathrm{head}_1,\dots,\mathrm{head}_h)\,W^O,$$8. Spanish is close to deterministic orthography-to-phonology under this evaluation, with LatPhon at 0.30% $$\mathrm{MultiHead}(Q,K,V) = \mathrm{Concat}(\mathrm{head}_1,\dots,\mathrm{head}_h)\,W^O,$$9, ByT5 at 0.25%, and WFST at 0.04% $$\mathrm{head}_i = \mathrm{Attention}(QW_i^Q,\,K W_i^K,\,V W_i^V),$$0. French is similarly low at 0.57% $$\mathrm{head}_i = \mathrm{Attention}(QW_i^Q,\,K W_i^K,\,V W_i^V),$$1, against 0.60% for ByT5 and 0.49% $$\mathrm{head}_i = \mathrm{Attention}(QW_i^Q,\,K W_i^K,\,V W_i^V),$$2 for WFST. Italian reverses the overall trend: LatPhon records 5.8% $$\mathrm{head}_i = \mathrm{Attention}(QW_i^Q,\,K W_i^K,\,V W_i^V),$$3, ByT5 3.1%, and WFST 5.4% $$\mathrm{head}_i = \mathrm{Attention}(QW_i^Q,\,K W_i^K,\,V W_i^V),$$4. Portuguese is particularly favorable to LatPhon, with 0.86% $$\mathrm{head}_i = \mathrm{Attention}(QW_i^Q,\,K W_i^K,\,V W_i^V),$$5 versus 9.1% for ByT5 and 2.7% $$\mathrm{head}_i = \mathrm{Attention}(QW_i^Q,\,K W_i^K,\,V W_i^V),$$6 for WFST. Romanian reaches 0.49% $$\mathrm{head}_i = \mathrm{Attention}(QW_i^Q,\,K W_i^K,\,V W_i^V),$$7, while WFST gives 0.23% $$\mathrm{head}_i = \mathrm{Attention}(QW_i^Q,\,K W_i^K,\,V W_i^V),$$8; no ByT5 result is reported.

The statistical analysis refines the aggregate ranking. LatPhon significantly outperforms ByT5 overall and beats WFSTs in Portuguese with $$\mathrm{head}_i = \mathrm{Attention}(QW_i^Q,\,K W_i^K,\,V W_i^V),$$9. In French and Italian, performance is statistically on par, with $$\mathrm{Attention}(Q,K,V) = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr)\,V.$$0. These results position LatPhon between two established baselines: it is substantially more compact than a large multilingual neural baseline and only modestly behind language-specific symbolic systems in mean PER. This suggests that its primary contribution lies in the efficiency-accuracy frontier rather than in absolute monolingual best performance.

5. Deployment profile and role in speech systems

The full fp32 checkpoint requires only 30 MB, and no additional quantization or pruning was applied to reach this size. At inference time, LatPhon decodes autoregressively at approximately 31 words/s on both a desktop 16-core CPU and an RTX 4090 GPU with batch size 1. The combination of sub-50 MB footprint and real-time speed is presented as sufficient for on-device voice applications (Chary et al., 3 Sep 2025).

The intended applications are explicit. In TTS, LatPhon reduces downstream decoder vocabularies to approximately 100 phonemes. In ASR, it provides consistent pronunciation mappings during lexicon building. In S2ST, it is intended to ensure cross-language phonetic consistency. In forced alignment pipelines, it supplies the phoneme sequences required for alignment to speech frames. Across these uses, the model functions as an interchangeable front-end whose main systems advantage is multilingual unification.

A related systems-level perspective appears in later work on low-latency, context-aware phonemization for real-time TTS. That work proposes a service-oriented architecture in which phonemization runs as a persistent background process queried by lightweight IPC, rather than as a sequentially loaded component inside a unified TTS runtime. On an Intel i7-1255U, the reported system achieves $$\mathrm{Attention}(Q,K,V) = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr)\,V.$$1 for “Piper + LCA-G2P,” while the latency for pipe I/O is reported as less than 1 ms (Fetrat et al., 8 Dec 2025). Although that architecture is not LatPhon itself, it clarifies the deployment environment in which compact phonemizers are valuable: low-latency, offline, and end-device TTS systems where cold-start and memory-mapping overheads matter as much as model quality.

6. Limitations, future directions, and terminological ambiguity

The limitations noted for LatPhon are concrete. Language coverage is restricted to Latin script, and extending to non-Latin alphabets or tonal languages would require additional IPA symbols and corpora. The model is sensitive to training-set size: Italian, with only approximately 6 k entries, yields the highest PER. Out-of-domain words such as neologisms and rare proper nouns may not be handled optimally because the training data originate from standard dictionary wordlists. Autoregressive decoding is described as a trade-off, exchanging ultimate throughput for simplicity and accuracy (Chary et al., 3 Sep 2025).

The stated future directions follow directly from those limitations: exploration of non-autoregressive variants for higher speed, co-training with acoustic models, synthetic data augmentation for low-resource languages, and broader language coverage. The code and models are to be released under an MIT license. In aggregate, these directions indicate that the present system is a deployable baseline rather than a closed design.

A recurring source of confusion is nominal rather than technical. The term “LatPhon” also appears in McLaughlin et al. as a label for LLaMA’s latent phonetic model: a structured internal phonetic representation inferred in a text-only transformer, analyzed through a $$\mathrm{Attention}(Q,K,V) = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr)\,V.$$2 linear phoneme probe, causal embedding interventions of the form $$\mathrm{Attention}(Q,K,V) = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr)\,V.$$3, and a “phoneme mover head” identified as H13L12 with $$\mathrm{Attention}(Q,K,V) = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr)\,V.$$4 (Merullo et al., 4 Aug 2025). That usage is distinct from the multilingual G2P system of (Chary et al., 3 Sep 2025). This suggests that “LatPhon” names two different objects in the 2025 literature: an explicit multilingual speech front-end and an emergent latent phonetic geometry in a LLM. The overlap is terminological; the methods, tasks, and evaluation regimes are separate.