Causal decomposition of LLM-induced speaker drift

Determine the causal contributions of LLM-induced drift, wording variation, semantic mismatch, training-distribution mismatch, and prompt-generation quality to speaker instability in prompt-conditioned Greek text-to-speech synthesis.

Background

The paper finds that LLM-generated style prompts introduce speaker drift in Parler-TTS, whereas deterministic prompts substantially improve speaker consistency, especially after speaker-specific LoRA adaptation. However, the observed instability may arise from several potentially interacting factors, including stochastic wording, semantic mismatch between prompts and speech, differences between training and inference prompt distributions, and the quality of the prompt-generation process. The authors explicitly leave the causal decomposition of these factors unresolved for future work.