Decoupling Conversational Dynamics in Full-Duplex Spoken Models through Reinforcement Learning

Abstract: Recent full-duplex spoken dialogue models have demonstrated compelling progress toward human-like interaction, enabling agents to respond with low latency, produce backchannels, and handle user barge-ins. Yet these improvements in conversational dynamics often come with weaker reasoning and instruction-following abilities, revealing a potential tension between interactive dynamics and intelligence capability. In this paper, we argue that such an intelligence--dynamics trade-off is not fundamental: conversational dynamics can instead be learned as a separate real-time decision policy from human dialogue data. To this end, we propose DuplexPO, a reinforcement learning (RL) framework that decouples when to speak from what to say. It preserves the semantic response capability of an instruction-tuned assistant, while optimizing its temporal interaction behavior over selected high-impact windows from long human conversations. To quantitatively optimize these dynamics, we formulate the Factorized Conversational Dynamics Reward (FCDR) to enable fine-grained temporal credit assignment for turn initiation, backchanneling, yielding, and regularized participation. The policy is then optimized with a GRPO-style objective. Experiments show that DuplexPO substantially improves full-duplex behaviors, including timely backchannels, smooth turn-taking, and barge-in handling, while maintaining strong reasoning and instruction-following performance. Moreover, improvements in dynamics-oriented metrics are reflected in better user experience, suggesting that optimizing conversational timing as a standalone objective can promote more natural full-duplex interaction.

• The paper shows that decoupling temporal cues from semantic content using an RL framework—DuplexPO—preserves high instruction following while improving conversational control.
• Methodologically, DuplexPO employs localized RL on dynamics-critical windows, yielding near-perfect turn-taking, backchanneling, and yield rates across datasets.
• Empirical results refute the trade-off hypothesis by demonstrating that enhanced timing does not compromise model intelligence in reasoning and task performance.

Decoupling Conversational Dynamics in Full-Duplex Spoken Dialogue Models via Reinforcement Learning

Introduction

The proliferation of full-duplex spoken LLMs (SDLMs) has significantly advanced the fidelity and naturalness of human-computer interaction. However, empirical evidence demonstrates a tension between conversational dynamics (such as low-latency floor control and interruption handling) and core instruction-following or reasoning capabilities. The paper "Decoupling Conversational Dynamics in Full-Duplex Spoken Models through Reinforcement Learning" (2607.07148) provides a detailed analysis of this trade-off and proposes a principled RL framework, DuplexPO, to decouple and independently optimize the temporal ("when to speak") and semantic ("what to say") modalities in full-duplex interactions.

Central Claim and Methodological Advances

The authors contend that the observed degradation in reasoning and intelligence benchmarks under full-duplex training is not due to an inherent modeling limitation but is primarily attributable to the monolithic coupling of content selection and interaction timing in existing objectives and datasets. DuplexPO operationalizes a two-factor decomposition: instruction-tuned SDLMs maintain semantic competence, while RL is applied exclusively to optimize temporal interaction behaviors—turn initiation, backchannel timing, yielding to user barge-in, and regularized participation—via the Factorized Conversational Dynamics Reward (FCDR).

DuplexPO samples dynamics-critical local windows from natural conversational data, restricting policy adaptation to temporally concentrated, high-salience events, rather than updating on full dialogues. This targeting is essential due to the local nature of conversational floor control cues.

Figure 1: Overview of the DuplexPO for full-duplex spoken dialogue models. The red shadows indicate dynamics-critical windows that may include important conversational behaviors, such as backchanneling, turn-taking, and user barge-in.

Architectural and Training Overview

The training protocol is staged: initial pretraining on synthetic and ASR-augmented speech data, followed by SFT for general instruction and task compliance, and finally RL fine-tuning confined to windowed conversational events. DuplexPO utilizes GRPO-style RL, normalizing rewards within each sampled window and regularizing updates toward the SFT reference model. This approach ensures behavioral improvements in floor control without drifting from the underlying semantic and reasoning capabilities learned during SFT.

The FCDR formulates explicit, interpretable rewards for turn-taking, backchanneling, and yielding, with fine-grained temporal credit assignment. This reward structure is critical for robust and transferable conversational policy learning, as the empirical analysis demonstrates (see discussion below).

Empirical Results

Conversational Dynamics

DuplexPO attains substantial improvements in all real-time conversational metrics. On Fisher and Seamless, DuplexPO achieves a turn-taking initiation rate and yield rate of 100.0%/98.7% and 98.0%/93.6%, respectively, with the lowest onset MAE recorded (e.g., 0.69s/1.03s). For backchanneling, DuplexPO delivers 97.8–99.5% initiation and 93.3–100% yield rates.

On the Full-Duplex-Bench v3 stress test—incorporating complex scenarios like barge-in and disfluency—DuplexPO yields the lowest turn-taking latency (0.24s), perfect yield rate (100.0%), and minimal voiced interrupt rate (5.0%), outperforming all strong public and research baselines.

Figure 2: Conversation-level Gemini pairwise evaluation comparing the SFT Baseline (A) with DuplexPO (B); the judge assesses turn-taking, backchanneling, and barge-in handling based on timestamps and aggregate dynamics statistics.

Subjective comparison via blinded LLM judge (Gemini 3.0 Pro) demonstrates a clear preference for DuplexPO over SFT, with 76.9% (Fisher) and 69.3% (Seamless) win rates, and especially pronounced gains in backchannel and barge-in dimensions.

Model Intelligence

Contrary to previous reports, DuplexPO preserves or slightly improves baseline SFT performance across factual QA (up to +3 points on Llama Questions), instruction following, speech understanding, and reasoning, as measured on VoiceBench, OpenBookQA, and MMSU. This falsifies the prior intelligence-dynamics trade-off hypothesis for decoupled, window-targeted RL.

Mechanistic Analysis

Suppression and release analysis (SIR/SRR metrics) and attention statistics confirm that DuplexPO does not globally bias toward more or less speaking. Instead, it localizes intervention around conversational boundaries, e.g., releasing <BOS>/<EOS> impulses selectively. Attention mass is shifted from static tokens (such as <BOS> sinks) to real-text and recent-context positions, indicating more dynamic use of conversational context.

Figure 3: Analysis of suppressed boundary intent in SFT Baseline and DuplexPO; (a) <EOS> impulse around user interruption events and (b) SIR/SRR rates, revealing selective release by DuplexPO.

Figure 4: Attention mass distribution averaged across layers and token categories. DuplexPO reduces <BOS> sink mass and increases attention to recent-context, real-text, and PAD states, evidencing improved context usage.

Figure 5: Example token-level conversational pattern. DuplexPO maintains and releases boundary impulses adaptively around user speech.

Ablation and Reward Analysis

Ablation studies establish that (i) training on extended windows away from event boundaries reduces RL reward, highlighting the necessity of tight temporal focus; (ii) FCDR-based rewards are more effective than neural reward models for conversational control; and (iii) GRPO yields denser floor-control learning than DPO-style objectives, particularly for yield-oriented behavior.

Figure 6: Training dynamics of the total reward and its individual components during RL, indicating stable policy convergence.

Figure 7: Reward component correlations with behavioral events, demonstrating specialisation of stop and start rewards.

Figure 8: Ablation of dynamics-critical window lead and buffer times: RL reward drops substantially for excessive anticipation windows, confirming the importance of local event proximity.

Implications and Future Directions

The DuplexPO approach refutes the notion that improved conversational responsiveness must degrade reasoning or task performance in SDLMs, provided real-time interaction policies are decoupled and optimized as a separate mechanism. This has both theoretical and engineering implications: high-level semantic functions and temporal coordination can be factorized and independently composed, which mirrors modular system architectures originally employed in traditional ASR-based pipelines.

Practically, DuplexPO's scalable, interpretable RL reward design can generalize across languages, speaker populations, and interaction regimes, with relatively minimal risk of semantic drift. However, potential limitations include FCDR's abstraction from subtle pragmatic factors (user intent, style) and the absence of explicit long-range or user-adaptive policy modeling. Integrating personalized or adaptive timing policies and extending evaluation to multilingual, multi-party, or adversarial turn-taking settings form natural directions for further research.

Conclusion

The paper demonstrates that the observed intelligence-dynamics trade-off in full-duplex spoken LLMs is not fundamental but arises from joint temporal-semantic objectives and training protocols. By decoupling conversational timing from semantic learning via DuplexPO and FCDR, SDLMs can simultaneously achieve natural, low-latency floor control and strong instruction/task performance. This result is strongly supported by both objective and subjective metrics and highlights the relevance of decomposed, targeted RL in future multimodal dialogue systems.