Optimising Neural Speech Codecs for 300bps Communication using Reinforcement Learning

Abstract: In bandwidth-constrained communication such as satellite and underwater channels, speech must often be transmitted at ultra-low bitrates where intelligibility is the primary objective. At such extreme compression levels, codecs trained with acoustic reconstruction losses tend to allocate bits to perceptual detail, leading to substantial degradation in word error rate (WER). This paper proposes ClariCodec, a neural speech codec operating at 300 bit per second (bps) that reformulates quantisation as a stochastic policy, enabling reinforcement learning (RL)-based optimisation of intelligibility. Specifically, the encoder is fine-tuned using WER-driven rewards while the acoustic reconstruction pipeline remains frozen. Even without RL, ClariCodec achieves 4.64% WER on the LibriSpeech test-clean set at 300 bps, already competitive with codecs operating at higher bitrates. Further RL fine-tuning reduces WER to 3.55% on test-clean and 10.4% on test-other, corresponding to a 23% relative reduction while preserving perceptual quality.

• The paper introduces ClariCodec, a 300 bps neural speech codec that treats stochastic residual quantisation as a policy and uses GRPO to optimise non-differentiable WER rewards.
• The codec achieves 3.55% WER on LibriSpeech test-clean and 10.4% on test-other, reducing WER by 23.5% and 21.8% through RL while maintaining a UTMOS of 4.16.
• The results show that an acoustic mel-loss anchor limits PESQ degradation during semantic optimisation, but the non-causal design, ASR-dependent reward, and lack of human evaluation remain important limitations.

Motivation and problem setting

In bandwidth-constrained channels such as satellite and underwater links, speech transmission may be limited to a few hundred bits per second, and the coding objective shifts from waveform fidelity to intelligibility. The authors argue that conventional neural codec training—mel-spectrogram $L_1$, adversarial, and feature matching losses—allocates scarce bits to perceptual detail rather than to the information required for linguistic decoding, since WER, the standard intelligibility metric, is discrete and non-differentiable. Framing this through the information bottleneck principle, they propose ClariCodec (2605.19541), a 300 bps neural speech codec that makes quantisation a stochastic policy so that reinforcement learning (RL) can directly optimise WER-based rewards. The paper claims two firsts: the lowest bitrate reported for a neural speech codec with competitive quality, and the first application of RL to neural speech codec training.

Architecture and training

ClariCodec operates on log-mel spectrograms with a 10 ms hop. A ConvNeXt V2-based encoder applies $8\times$ temporal downsampling to yield a 12.5 Hz latent frame rate; a symmetric decoder reconstructs the spectrogram, and a Vocos vocoder trained jointly converts it to waveforms. Quantisation uses residual FSQ (R-FSQ) with two layers of level configuration $\mathcal{L}=[8,8,8,8]$ (12 bits per layer), giving $12.5 \times 24 = 300$ bps. Improved FSQ replaces the tanh bounding function with sigmoid activation to improve codebook utilisation.

The key architectural modification is stochastic residual quantisation: instead of deterministic rounding, negative squared distances to grid points serve as logits, and levels are sampled via Gumbel-Softmax. This renders the quantiser a differentiable policy $\pi_\theta$ amenable to policy-gradient optimisation.

Training proceeds in two stages. Stage 1 performs end-to-end reconstruction pre-training with an $L_1$ mel loss plus Hinge-GAN adversarial losses over multi-period, multi-resolution, and multi-scale discriminators, together with feature matching. Stage 2 freezes everything except the encoder and fine-tunes it with group relative policy optimisation (GRPO): for each input, $G=16$ token sequences are sampled, sequence-level rewards are the negative WER between ASR transcriptions of reconstructed and ground-truth audio (computed by a 1.1B-parameter Hybrid FastConformer TDT-CTC model), and advantages are group-normalised. Because KL regularisation against a reference policy is incompatible with the unidirectional encoder, an $L_1$ mel reconstruction loss is used instead as an acoustic anchor.

Results

Trained on 50k hours of Libriheavy and evaluated on LibriSpeech test-clean/test-other, ClariCodec achieves strong results at the lowest bitrate among compared systems:

Model	bps	test-clean WER (%)	test-other WER (%)	UTMOS (clean)
EnCodec	750	16.1	36.4	1.25
StableCodec-400	400	4.88	14.4	4.31
SemantiCodec	312.5	22.7	40.2	2.72
SAC	525	2.00	4.15	4.27
FlexiCodec	640	2.57	4.69	4.15
ClariCodec (w/o RL)	300	4.64	13.3	4.12
ClariCodec (full)	300	3.55	10.4	4.16

Even before RL, the codec attains 4.64% WER on test-clean, competitive with systems at substantially higher bitrates—for example outperforming StableCodec-400 (4.88%) while using 25% fewer bits. RL fine-tuning reduces WER by 23.5% relative on test-clean and 21.8% on test-other, demonstrating generalisation beyond clean conditions, while UTMOS slightly improves (4.12 → 4.16). The ablation shows that unconstrained RL yields nearly identical WER gains (3.54%) but degrades PESQ from 1.88 to 1.83; adding the mel anchor restores PESQ to 1.87 while retaining the intelligibility benefit, though not fully recovering the Stage 1 level—an acknowledged trade-off between acoustic fidelity and semantic optimisation at extreme compression.

Limitations and open questions

Several caveats bear directly on these results. First, the architecture is non-causal, so the system is not yet suitable for real-time streaming communication—the primary use case motivating ultra-low-bitrate coding—and the authors identify a low-latency streaming variant as future work. Second, the reward is computed by a specific ASR model, raising the possibility of reward over-optimisation toward that recogniser's biases; the paper does not evaluate robustness across multiple ASR systems or against human listening tests, relying instead on objective metrics (WER, STOI, PESQ, UTMOS, SIM). Third, the mel anchor only partially recovers PESQ, leaving open how to design reward functions that jointly optimise intelligibility and acoustic quality. Finally, the effect of such aggressively compressed tokens on downstream generative tasks—codec-based speech LLMs and synthesis—is unexamined.

Conclusion

ClariCodec demonstrates that recasting discrete codec quantisation as a stochastic policy enables GRPO-based optimisation of non-differentiable intelligibility objectives within a frozen acoustic pipeline. At 300 bps it achieves 3.55% WER on LibriSpeech test-clean, surpassing codecs operating at up to 1.75× its bitrate on this metric while maintaining competitive UTMOS, establishing RL-based semantic optimisation as a viable complement to reconstruction-driven codec training.