Optimize TTS Models with D-LPJ Vector Rewards

Investigate the use of Decoupled-Live-ProsodyJudge’s dimension-wise vector rewards to optimize a text-to-speech model through post-training.

Background

Decoupled-Live-ProsodyJudge (D-LPJ) evaluates four core prosodic dimensions—fluency and naturalness, intonation variation, emotional expression, and live-streaming expressiveness—producing separate dimension-level judgments rather than a single overall verdict. The paper demonstrates that these vector judgments can support diagnosis and external scalar decisions, but it does not use them as rewards to train or optimize a TTS system.

The unresolved problem is to determine how dimension-wise D-LPJ rewards can be incorporated into TTS post-training, such as reinforcement learning or another optimization procedure, while preserving the intended independence of the prosodic dimensions and avoiding renewed verdict coupling.

References

D-LPJ is currently limited to C1--C4, and using its vector rewards to optimize a TTS model remains future work.

— Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis  (2609.20124 - Guan et al., 17 Sep 2026) in Section 5, “Best-of-8 and limitations”