DuplexPO: Decoupling Full-Duplex Dialogue

• DuplexPO is a reinforcement learning framework that decouples conversational timing from content generation to address the intelligence–dynamics trade-off.
• It employs a dual architecture with an instruction-tuned semantic model and a real-time dynamics policy optimized via localized interaction-critical windows.
• Empirical results show that DuplexPO improves onset accuracy, turn yield, and backchannel timing while maintaining strong semantic performance.

DuplexPO is a reinforcement learning framework for full-duplex spoken dialogue models that decouples “when to speak” from “what to say,” with the explicit aim of improving conversational dynamics without degrading the semantic response capability of an instruction-tuned assistant. It is introduced in “Decoupling Conversational Dynamics in Full-Duplex Spoken Models through Reinforcement Learning” (Li et al., 8 Jul 2026), which argues that the apparent intelligence–dynamics trade-off in recent full-duplex spoken systems is not fundamental. Instead, the paper treats conversational timing as a separate real-time decision policy learned from human dialogue data and optimized over localized interaction-critical windows, while preserving the assistant’s reasoning, instruction-following, and speech-understanding behavior (Li et al., 8 Jul 2026).

1. Concept and problem setting

Full-duplex spoken dialogue models enable agents to listen and speak simultaneously, supporting low latency, timely backchannels, and user barge-in handling. The DuplexPO paper identifies a recurrent difficulty in this setting: recent full-duplex models often degrade on instruction-following and reasoning benchmarks relative to half-duplex or text-first assistants, yielding an apparent intelligence–dynamics trade-off (Li et al., 8 Jul 2026).

The paper attributes this tension to both modeling and data. On the modeling side, strong reasoning requires long, structured decoding with multi-step inference, whereas natural full-duplex interaction requires continuously processing fragmented, rapidly changing audio contexts and making local real-time decisions such as start, pause, and yield. On the data side, dynamics learning via SFT often uses casual conversational corpora such as Fisher, which are not aligned with helpful assistant behavior, so optimizing raw conversational imitation can harm instruction-following competencies learned elsewhere (Li et al., 8 Jul 2026).

DuplexPO is built around the paper’s central hypothesis: this trade-off is not fundamental. The proposed remedy is architectural and objective-level decoupling. “When to speak” is cast as real-time floor control, optimized as an independent decision policy over high-impact windows in human conversations, while “what to say” remains anchored in an instruction-tuned assistant. This formulation targets turn-taking, backchanneling, yielding, and participation regularization without re-learning global semantic behavior from casual dialogue data (Li et al., 8 Jul 2026).

A plausible implication is that DuplexPO reframes full-duplexity as a control problem rather than a content-generation problem. In the paper’s terms, the optimization target is temporal interaction behavior, not content imitation.

2. Decoupled architecture and frame-level formulation

DuplexPO uses a decoupled architecture with two components: a semantic response model and a real-time dynamics policy. The semantic response model is an instruction-tuned assistant that retains “what to say” competencies via pre-training and SFT on QA, instruction-following, and speech understanding tasks. The speech encoder and codec remain frozen to preserve learned representations, and streaming TTS is trained after RL for speech generation. The real-time dynamics policy is a frame-level decision policy optimized via RL over localized interaction-critical windows; it decides “when to speak” through actions for turn initiation, backchannel production, yielding after user barge-in, and participation regularization (Li et al., 8 Jul 2026).

The full-duplex conversational dynamics policy is defined on a uniform temporal grid with frame duration $\Delta$. Let $x_{1:T}$ be the user audio stream and $y_{1:T}$ the agent’s real-time interaction decisions. The causal streaming policy is

$\pi_\theta(y_t \mid y_{<t}, x_{\le t}).$

Here $y_t$ is an interaction state such as silence, speaking, turn initiation (<BOS>), or yielding (<EOS>) (Li et al., 8 Jul 2026).

The state $s_t$ implicitly comprises the LLM’s streaming context over acoustic features mapped into the embedding space by a Parakeet-based speech encoder and a transformer modality adapter. The agent maintains frame-synchronous decisions at 12.5 Hz, while the audio codec operates at 25 Hz. Actions are realized through special text tokens and PAD positions, and the text head’s decisions gate TTS output. The system does not rely on an explicit ASR token stream for timing; instead, prosodic cues and word-level timing influence the policy through streaming feature representations (Li et al., 8 Jul 2026).

A distinctive component is Dynamics-Critical Window Sampling. DuplexPO does not optimize over entire dialogues. It extracts local windows around annotated agent segments in human conversations where timing decisions are most impactful, specifically turn transitions, backchannels, and user barge-ins. For an annotated human agent segment with start–end times $(a_i,b_i)$ and metadata $c_i$, the discretized reference interval is

$$g_i = \lfloor a_i / \Delta \rfloor,\qquad h_i = \lfloor b_i / \Delta \rfloor.$$

The window $W_i=(s_i,e_i,g_i,h_i,c_i)$ includes lead time $x_{1:T}$0 before $x_{1:T}$1 and buffer time $x_{1:T}$2 after $x_{1:T}$3, with boundary clipping:

$x_{1:T}$4

$x_{1:T}$5

The trajectory before $x_{1:T}$6 is teacher-forced, and actions are sampled and optimized only within $x_{1:T}$7 (Li et al., 8 Jul 2026).

This local-window construction is central to the framework’s claim that temporal credit assignment can be improved without perturbing long-range semantic behavior.

3. Factorized Conversational Dynamics Reward

To optimize timing, DuplexPO introduces the Factorized Conversational Dynamics Reward (FCDR), a weighted sum of factorized components aligned to human timing annotations. For a sampled continuation inside window $x_{1:T}$8, let $x_{1:T}$9 denote frames where the agent speaks. If $y_{1:T}$0, the predicted onset is $y_{1:T}$1, and the onset delay is

$y_{1:T}$2

with $y_{1:T}$3 for early starts and $y_{1:T}$4 for late starts. Event metadata activates masks $y_{1:T}$5, $y_{1:T}$6, and $y_{1:T}$7 for onset timing, backchannel timing, and barge-in yielding, respectively (Li et al., 8 Jul 2026).

The reward is

$y_{1:T}$8

The turn-initiation component is

$y_{1:T}$9

with range $\pi_\theta(y_t \mid y_{<t}, x_{\le t}).$0. The paper states that $\pi_\theta(y_t \mid y_{<t}, x_{\le t}).$1 shapes tolerance, and early starts are typically penalized more sharply than mild delays (Li et al., 8 Jul 2026).

For backchanneling, if $\pi_\theta(y_t \mid y_{<t}, x_{\le t}).$2 is the annotated backchannel interval and $\pi_\theta(y_t \mid y_{<t}, x_{\le t}).$3 the frame distance from $\pi_\theta(y_t \mid y_{<t}, x_{\le t}).$4 to $\pi_\theta(y_t \mid y_{<t}, x_{\le t}).$5, then

$\pi_\theta(y_t \mid y_{<t}, x_{\le t}).$6

again with range $\pi_\theta(y_t \mid y_{<t}, x_{\le t}).$7. This rewards near-interval backchannel timing and penalizes stale or floor-grabbing behavior (Li et al., 8 Jul 2026).

For yielding after barge-in, if $\pi_\theta(y_t \mid y_{<t}, x_{\le t}).$8 is the duration of agent speech after user barge-in, $\pi_\theta(y_t \mid y_{<t}, x_{\le t}).$9 a target maximum, and $y_t$0 a normalizing scale, then

$y_t$1

with range $y_t$2. Pattern regularization is

$y_t$3

where undesirable interactions include false alarms, near-target PAD stalls, and excessive overlap (Li et al., 8 Jul 2026).

The factorization is important because rewards are computed only inside dynamics-critical windows and activated through event-specific masks derived from annotated segments. This avoids direct content imitation and instead assigns credit to turn initiation, backchannel timing, stop timing, and interaction regularization. The paper’s ablations report that, holding sampler and GRPO fixed, FCDR outperforms a Neural Reward Model on Fisher and Seamless window metrics and on FDB-v3 yield. For example, Fisher Onset MAE is 0.69 for FCDR versus 1.22 for NRM, and Fisher Backchannel Yield is 100.0 versus 50.0; on Seamless, Turn Yield is 93.6 versus 67.6, and FDB-v3 Yield is 100.0% versus 93.3% (Li et al., 8 Jul 2026).

4. Optimization, training pipeline, and implementation

DuplexPO uses a GRPO-style objective over sampled continuations inside each window. For window $y_t$4, $y_t$5 sampled continuations yield rewards $y_t$6, and group-normalized clipped advantages are

$y_t$7

where $y_t$8 and $y_t$9 are the group mean and standard deviation across the $s_t$0 continuations. The optimization objective is

$s_t$1

with

$s_t$2

The paper emphasizes that advantages are computed at the window level via group normalization rather than temporal bootstrapping, and that no separate value function baseline or TD($s_t$3) is used (Li et al., 8 Jul 2026).

The described training loop samples a conversation, reconstructs segments with word-level timing, annotates metadata such as backchannel flag and barge-in presence, builds dynamics-critical windows with lead $s_t$4 and buffer $s_t$5, teacher-forces the history up to $s_t$6, samples $s_t$7 continuations under the current policy with temperature/top-$s_t$8, computes FCDR, normalizes rewards to advantages, applies $s_t$9 and $(a_i,b_i)$0, and updates parameters with AdamW (Li et al., 8 Jul 2026).

The implementation uses Qwen2.5-7B-Instruct as the LLM backbone, a 600M-parameter Parakeet-based speech encoder with causal convolutional context, and a 1024-dimensional Transformer modality adapter. Speech generation uses a CosyVoice2 streaming flow-matching generator. The speech encoder and codec are frozen. The audio codec runs at 25 Hz, the LLM at 12.5 Hz, and each LLM frame predicts two speech tokens to keep streams aligned. Real-time timing decisions are enacted by PAD during silence, <BOS> for turn initiation, and <EOS> for yielding; an <EOS> masks audio output to implement barge-in handling and turn release (Li et al., 8 Jul 2026).

Training uses 64 A800 80 GB GPUs, bfloat16 mixed precision, and gradient norm clip 1.0. The RL configuration uses $(a_i,b_i)$1 s, $(a_i,b_i)$2 s, $(a_i,b_i)$3 s, validation lead/buffer 2.0/2.0 s, up to 3 full-turn and 1 backchannel window per conversation, $(a_i,b_i)$4 samples per window, temperature/top-$(a_i,b_i)$5, max rollout steps 200, chunk size 128, AdamW learning rate $(a_i,b_i)$6, warm-up 100 steps, minimum learning rate $(a_i,b_i)$7, inverse-square-root schedule, KL coefficient $(a_i,b_i)$8, and advantage clipping $(a_i,b_i)$9 (Li et al., 8 Jul 2026).

The training data sources include 530K hours of speech-continuation pre-training, 70K hours of spoken QA, ASR-QA, synthetic user interruption data, and dynamics-aware dialogue reconstruction from Fisher and Seamless with word-level alignments. The sampled training set contains 24.6K Fisher and 43.1K Seamless samples, with conversation/session-level splits fixed before reconstruction and per-sample truncation at 210 s. Acoustic augmentation combines SpecAugment with waveform mixing using 10K noise clips from Freesound and MUSAN at $c_i$0 and SNR uniformly sampled from 0 to 60 dB (Li et al., 8 Jul 2026).

5. Empirical behavior, user study, and semantic preservation

The evaluation separates dynamics-oriented metrics from intelligence-oriented benchmarks. Dynamics metrics include Window-level initiation rate, Onset MAE, Yield rate, and FDB-v3 measures: Turn-Taking Latency, Voiced Interrupt Rate (VIR), and Yield Rate. Comparison systems include commercial full-duplex agents, open-source duplex models, SFT Baseline, and SFT Dynamics; the paper states that content models are held fixed across DuplexPO and SFT controls and that DuplexPO optimizes timing only (Li et al., 8 Jul 2026).

The reported window-level results on Fisher and Seamless show large gains over the two SFT controls.

Setting	SFT Baseline	SFT Dynamics	DuplexPO
Fisher Onset MAE	0.98	1.14	0.69
Fisher Turn Yield	92.1	78.9	98.7
Fisher Backchannel Yield	57.1	57.1	100.0
Seamless Onset MAE	1.22	1.34	1.03
Seamless Turn Yield	78.5	63.0	93.6
Seamless Backchannel Yield	79.4	63.4	93.3

On Fisher, DuplexPO reports Onset MAE 0.69, Turn Init 100.0, Turn Yield 98.7, Backchannel Init 97.8, and Backchannel Yield 100.0, compared with SFT Baseline values of 0.98, 97.8, 92.1, 95.7, and 57.1. On Seamless, DuplexPO reports 1.03, 98.0, 93.6, 99.5, and 93.3, compared with SFT Baseline values of 1.22, 91.8, 78.5, 92.2, and 79.4 (Li et al., 8 Jul 2026).

FDB-v3 further isolates real-time interaction behavior. DuplexPO reports Turn-Take 100.0%, Latency 0.24 s, VIR 5.0%, and Yield 100.0%. The same benchmark reports 99.0%, 7.33 s, 8.0%, and 64.8% for SFT Baseline; 100.0%, 14.24 s, 4.0%, and 93.3% for SFT Dynamics; 100.0%, 0.44 s, 12.0%, and 71.9% for Moshi; and 98.0%, 1.41 s, 15.0%, and 66.7% for PersonaPlex (Li et al., 8 Jul 2026). The paper explicitly interprets these numbers as breaking the latency-versus-interruption trade-off: DuplexPO attains 0.24 s latency, 100% Yield Rate, and 5% VIR (Li et al., 8 Jul 2026).

The user study uses Gemini 3.0 Pro as an LLM-as-a-Judge under a blinded matched-window protocol. DuplexPO is preferred on Fisher with 76.9% overall preference ($c_i$1), 76.9% for turn-taking ($c_i$2), 91.7% for backchannel ($c_i$3), and 87.5% for barge-in handling ($c_i$4). On Seamless, the reported preferences are 69.3% overall ($c_i$5), 67.0% for turn-taking ($c_i$6), 82.9% for backchannel ($c_i$7), and 77.2% for barge-in handling ($c_i$8). Judge calibration using synthetic timing perturbations reports original win rate 80.8% versus advanced by $c_i$9 s ($$g_i = \lfloor a_i / \Delta \rfloor,\qquad h_i = \lfloor b_i / \Delta \rfloor.$$0) and 69.2% versus delayed by $$g_i = \lfloor a_i / \Delta \rfloor,\qquad h_i = \lfloor b_i / \Delta \rfloor.$$1 s ($$g_i = \lfloor a_i / \Delta \rfloor,\qquad h_i = \lfloor b_i / \Delta \rfloor.$$2), indicating sensitivity to timing (Li et al., 8 Jul 2026).

A separate claim of the framework is semantic preservation. The paper reports that DuplexPO maintains or slightly improves over SFT Baseline across factual QA, instruction following, speech understanding, and reasoning benchmarks: Llama Questions 75.3 versus 72.0, WebQuestions 44.5 versus 44.3, TriviaQA 49.9 versus 48.1, SDQA 49.8 versus 47.2, AlpacaEval 3.68 versus 3.43, CommonEval 3.74 versus 3.48, OpenBookQA 73.7 versus 72.2, and MMSU 56.2 versus 54.9 (Li et al., 8 Jul 2026). This supports the paper’s claim that timing-only RL can improve conversational dynamics while preserving “what to say.”

The qualitative analysis is consistent with these metrics. Token-level visualization shows that DuplexPO releases suppressed <BOS> and <EOS> impulses selectively around user speech instead of remaining near PAD, and attention analysis reports less mass on the initial <BOS> sink and more on real-text, recent-context, and PAD-state positions than Moshi 7B. Appendix metrics report, for Fisher, <BOS> SIR 11.0% with SRR 70.6% and <EOS> SIR 4.9% with SRR 100.0%; on FDB-v3, <BOS> SIR 10.8% with SRR 71.9% and <EOS> SIR 0.9% with SRR 100.0% (Li et al., 8 Jul 2026).

6. Positioning, limitations, and future directions

The paper positions DuplexPO against both end-to-end full-duplex models and system-level heuristics. Compared to end-to-end full-duplex models such as Moshi, SALMONN-omni, and SALM-Duplex, and compared to heuristics such as VAD and dialogue management, DuplexPO treats model-level full-duplexity as learning an explicit timing policy. Its defining move is to optimize real-time floor-control decisions through RL over dynamics-critical windows rather than re-learning complete dialogue responses dominated by casual exchanges (Li et al., 8 Jul 2026).

The ablation results sharpen that positioning. Longer lead times reduce mean RL reward sharply, while buffer time has milder effects, suggesting that supervision must stay close to boundary events. GRPO outperforms a DPO-style preference objective on yield and backchannel metrics: on Fisher, Onset MAE is 0.69 versus 0.81, Turn Yield 98.7 versus 93.6, and Backchannel Yield 100.0 versus 71.4; on Seamless, Turn Yield is 93.6 versus 77.8 and Backchannel Yield 93.3 versus 84.7; FDB-v3 Yield is 100.0% versus 93.3% (Li et al., 8 Jul 2026). The paper explains this by stating that GRPO leverages full reward distributions via group-normalized advantages, whereas DPO discards ranking information by focusing only on extremes.

The reported limitations are specific. FCDR may miss pragmatic subtleties such as intent, speaker style, and culture-specific timing; human timing annotations are not unique ground truth; local window optimization may miss long-range effects; and the system remains sensitive to noisy environments, fast speakers, and ambiguous turn transitions despite augmentation (Li et al., 8 Jul 2026). This suggests that the framework is strongest when conversational boundary events are locally identifiable from streaming acoustic context, and less decisive when dialogue structure depends on subtle pragmatic interpretation.

The ethics discussion is correspondingly narrow but concrete: more human-like voice dynamics can be misused, including for vishing, and safeguards such as watermarking and identity disclosure should be considered (Li et al., 8 Jul 2026). Future work is described as multilingual and user-adaptive evaluation, longer-horizon coherence objectives, integrating richer prosodic and semantic cues, and combining factorized rewards with learned temporal reward models (Li et al., 8 Jul 2026).

In aggregate, DuplexPO defines a specific research program inside spoken dialogue modeling: semantic generation remains instruction-tuned and reference-anchored, while turn-taking, backchanneling, yielding, and participation are optimized as an explicit real-time policy. The paper’s empirical results and ablations support its central claim that the intelligence–dynamics trade-off is largely an artifact of coupled training objectives rather than a fundamental conflict (Li et al., 8 Jul 2026).