Praxy Voice: Voice-Prompt Recovery + BUPS for Commercial-Class Indic TTS from a Frozen Non-Indic Base at Zero Commercial-Training-Data Cost

Abstract: Commercial TTS systems produce near-native Indic audio, but the best open-source bases (Chatterbox, Indic Parler-TTS, IndicF5) trail them on measured phonological dimensions, and the most widely adopted multilingual base (Chatterbox, 23 languages) does not even tokenise Telugu or Tamil. We ask: what is the minimum intervention that brings such a non-Indic-native base to commercial-class output on Telugu, Tamil, and Hindi, without training a new acoustic decoder and without any commercial TTS training data? We combine three pieces: (1) BUPS, a Brahmic Unified Phoneme Space that deterministically romanises seven Indic scripts to ISO-15919 so Chatterbox's Latin tokeniser can process them; (2) a LoRA adapter on only the text-token predictor (Chatterbox's t3), trained on ~1,220h of licensed Indic audio with a Hindi-proxy language_id; (3) a voice-prompt recovery recipe -- an 8-11s same-language reference clip plus three sampling overrides (exaggeration 0.7, temperature 0.6, min_p 0.1; "Config B") -- that recovers commercial-class acoustic output with no acoustic-decoder training. On Hindi, the LoRA regresses accuracy and we instead use vanilla Chatterbox + Config B, giving a two-branch deployment. Evaluated on 10-utterance pilot sets with the companion PSP benchmark, Praxy Voice matches or slightly leads commercial baselines: 26.7% retroflex collapse on Telugu (vs Sarvam Bulbul 33.3%), 71% Tamil-zha collapse (vs commercial trio's 86%), 0.025 LLM-WER on Hindi (tied with Cartesia Sonic-3). For intra-sentential code-mix we add a third branch (IndicF5 + native-script transliteration) that drops code-mix LLM-WER from 0.80-0.85 to 0.14-0.27 across Hi/Te/Ta. We release R6 LoRA weights (Apache-2.0), inference code and router (MIT), and a Gradio demo.

• The paper introduces minimal parameter-efficient interventions to extend a non-Indic TTS model for commercial-grade Indic speech without proprietary data.
• It combines BUPS romanization, tailored LoRA adaptation, and voice-prompt recovery to enhance phonological fidelity and intelligibility.
• Empirical results demonstrate significant gains in accent accuracy and naturalness, validating the approach for low-resource language TTS.

Minimum-Intervention Adaptation for Indic TTS: Summary of "Praxy Voice" (2604.25441)

Introduction and Motivation

The "Praxy Voice" system proposes an efficient methodology to adapt a large-scale, open-source, non-Indic-native multilingual text-to-speech (TTS) model (ResembleAI Chatterbox) to high-quality, commercial-class production of Telugu, Tamil, and Hindi speech, crucially without any proprietary or commercial TTS data and without acoustic model retraining. The paper addresses the bottleneck that most open-source TTS bases, including those with broad multilingual coverage, either lack native support for major Indic scripts or perform significantly behind commercial APIs on critical phonological and intelligibility axes. Training a purpose-built Indic TTS from scratch, as in AI4Bharat's models or OmniVoice, remains resource-intensive and impractical for most production use-cases.

Methodological Innovations

Three main engineering interventions are introduced and composed into a unified recipe:

1. BUPS (Brahmic Unified Phoneme Space):

A deterministic romanization front-end based on ISO-15919 maps Devanagari, Telugu, Tamil, and other Brahmic scripts to Latin-script phoneme representations, allowing the frozen Chatterbox's Latin tokeniser to process these low-resource languages without modification or additional tokenisation. This mapping ensures phonemic fidelity and is lossless for the supported scripts.

2. Targeted LoRA Adaptation:

A LoRA adapter (∼7.86M parameters, <1% of base) is trained exclusively on Chatterbox's text-token predictor ($t_3$), leveraging BUPS-preprocessed Indic text and using a Hindi-proxy language ID to exploit the acoustic manifold closest to the target languages. The core acoustic generation components (the acoustic decoder $s_3$gen and voice encoder ve) remain frozen, maintaining compute efficiency.

3. Voice-Prompt Recovery with Config B Sampling:

The inference pipeline includes voice cloning via an 8–11s same-language prompt, with specific sampling overrides (exaggeration=0.7, temperature=0.6, min_p=0.1—Config B). This compensates for acoustic mismatch by steering the frozen acoustic decoder onto the correct prosodic and phonological manifold, based on empirical ablation studies.

Additionally, for intra-utterance code-mixing—a pervasive phenomenon in Indian speech text—inputs are routed to a transliteration preprocessor (via an instruction-tuned LM) and then processed by IndicF5, a character-level open-source Indic TTS.

Empirical Results and Analysis

Phonological and Intelligibility Benchmarks

Using the companion PSP benchmark [psp2026], which evaluates detailed phoneme-level and distributional accent metrics, "Praxy Voice" demonstrates the following headline results:

• Telugu: Retroflex collapse 26.7% (vs Sarvam Bulbul 33.3%). Ranking is first though the difference falls within the small-sample noise band.
• Tamil: Tamil-zha (\b{r}) collapse at 71% (vs commercial systems 86%), the clearest per-feature gain.
• Hindi: LLM-WER of 0.025, tying with Cartesia Sonic-3.

All strong results are obtained with the acoustic model remaining frozen and no commercial or proprietary TTS training data.

Scope Control and Deployment Architecture

A negative control on Hindi is critical: engaging the LoRA pathway regresses semantic accuracy (LLM-WER inflates from 0.025 to 0.334), affirming that the adaptation pathway is beneficial solely for languages outside the base's native set. This leads to a deployment split:

• LoRA+BUPS for Telugu/Tamil
• Vanilla Chatterbox for Hindi
• Code-mix branch (transliteration + IndicF5) for code-switched utterances

Robustness and Ablations

Config B Sampling

Empirical ablation shows that the triple-override (exaggeration, temperature, min_p) dominates alternatives in LLM-WER and FAD, with listener trials corroborating superior perceived prosody and naturalness.

Reference-Audio Sourcing

Using same-language voice prompts achieves materially lower FAD and more native prosody; cross-language prompts (e.g., English reference for Telugu) harm output quality. This validates the same-language constraint for reference audio.

Training Scale

Scaling from 85h to 1220h of training data for the LoRA branch improved LLM-WER by 5x, with retroflex collapse metrics essentially unchanged—a sign that token-level adaptation is effective independently of acoustic retraining.

Code-Mix Handling

The transliteration preprocessor feeding IndicF5 reduces LLM-WER on code-mixed Telugu from 0.80 to 0.14, closing much of the gap to commercial systems (Cartesia Sonic-3 at 0.11). For Hindi, the reduction is from 0.85 to 0.20; the remaining metric gap largely reflects differences in STT training data and handling of code-mixed pronunciations.

Theoretical and Practical Implications

The principal contribution is demonstrating how minimal parameter-efficient intervention can extend a frozen non-Indic base to commercial-class performance on phonetically complex, low-resource Indic languages, obviating the need for expensive acoustic retraining or commercial data licensing. Applying BUPS as a romanisation-based routing mechanism enables any adequately trained Latin-tokenised TTS base to bootstrap new Brahmic-script languages with minimal cost.

Performance parity or superiority on per-phoneme and semantic accuracy metrics, matched with open-source, commercially viable code and weights, stands to democratise high-quality Indic TTS for a broad range of production and low-resource research settings.

Limitations and Future Work

Key limitations include reliance on single-digit test set sizes (pilot10), the inability to LoRA-adapt the acoustic decoder due to resource constraints, and the absence of formal MOS evaluation. Acoustic-decoder LoRA, larger-scale subjective evaluation, and further fine-tuning on naturally sourced code-mix data are proposed as next steps, supported by imminent access to more substantial GPU resources. Notably, the code-mix branch, while closing the WER gap for Telugu, still lags on Hindi due to STT model alignment with American English code-mixing.

Conclusion

"Praxy Voice" (2604.25441) establishes that a frozen, open-source, non-Indic TTS base can be adapted to robustly cover major Indic languages using a minimal, parameter-efficient intervention. Key technical advances include BUPS for romanisation-routing, narrowly-scoped LoRA fine-tuning, and a validated inference-time voice-prompt recipe. The empirical approach—combining strong accent and intelligibility results across Telugu, Tamil, and Hindi—demonstrates parity or marginal superiority versus closed-source commercial APIs under zero proprietary training data constraints. This approach offers actionable methodology for extending production TTS to other underserved orthographic and phonological domains with minimal compute or data investment.