DualTurn: Dual-Channel Turn-Taking Model

• DualTurn is a dual-channel turn-taking model that uses generative pretraining on audio to learn conversational structure and anticipatory signals without human labels.
• It employs a two-stage approach with autoregressive audio prediction followed by fine-tuning for six self-supervised signals that map directly to interpretable agent actions.
• Experimental results demonstrate improved turn prediction, reduced misfires, and earlier boundary anticipation compared to methods like VAP across multiple benchmarks.

Searching arXiv for papers on DualTurn and closely related turn-taking work to ground the article. DualTurn is a label that appears in several technical literatures, but one prominent usage denotes a turn-taking model for spoken dialogue systems that uses dual-channel generative speech pretraining to transfer anticipatory conversational behavior into modular ASR–LLM–TTS pipelines (Rajaa, 9 Mar 2026). In that usage, the model generates both speakers’ future audio autoregressively, learns conversational dynamics without human turn-taking labels, and is then fine-tuned to predict interpretable turn-taking signals that map directly to agent actions. The label also appears in a distinct adherent-cell migration literature as a “runner-rotor” picture of bimodal motion (Zhang et al., 26 Mar 2025), while closely related recent work extends turn-aware reasoning to multiparty audio, multi-turn dialogue safety, and tool-calling agents (Patamia et al., 15 Jun 2026).

1. Dual-channel turn-taking as the main DualTurn formulation

The conversational-AI formulation of DualTurn was introduced to address a specific production tradeoff. Speech-to-speech models handle turn-taking naturally because they operate directly on audio and are trained to predict future speech in both channels, so they implicitly model overlaps, backchannels, pauses, and turn shifts, but they offer limited support for tool-calling or complex reasoning. Production ASR–LLM–TTS voice pipelines have those reasoning capabilities, but their turn-taking is usually handled by silence timeouts or VAD-triggered endpointing, which makes them reactive rather than predictive (Rajaa, 9 Mar 2026).

DualTurn narrows this gap through generative pretraining on dual-channel conversational audio. Its core premise is that if a model learns to predict what each speaker will say next in a dialogue, it implicitly learns conversational structure, timing, and interaction patterns. This turns turn-taking into a representation-learning problem rather than a hand-labeled classification problem. The model explicitly monitors both speakers’ channels simultaneously and is trained first on raw conversational audio without labels, then fine-tuned to predict self-supervised turn-taking signals that map directly to agent actions (Rajaa, 9 Mar 2026).

This design differs from several earlier families of methods. Silence-timeout pipelines do not determine whether a pause is a true turn end, a brief hesitation, or a backchannel opportunity. Text-only turn-taking models miss prosody and timing. Audio-only classifiers are often single-channel and binary. Voice Activity Projection (VAP) is the closest prior work because it is self-supervised and dual-channel, but it predicts future voice activity rather than a richer action space and cannot clearly distinguish turn ends from backchannels or other conversational events (Rajaa, 9 Mar 2026).

2. Training setup, signal taxonomy, and model architecture

DualTurn has two stages. In Stage 1, the model is pretrained autoregressively on both speakers’ future audio. Using Mimi codec embeddings, it observes past dual-channel audio and predicts next audio tokens or frames for both channels simultaneously. The paper writes the generative objective as

$$\mathcal{L}_{\text{gen}} = - \sum_t \left[ \log p_\theta(x^u_{t+1} \mid x^u_{\le t}, x^a_{\le t}) + \log p_\theta(x^a_{t+1} \mid x^u_{\le t}, x^a_{\le t}) \right].$$

After this pretraining stage, the depth predictor used for autoregressive generation is discarded and the representation backbone is retained for turn-taking prediction (Rajaa, 9 Mar 2026).

In Stage 2, DualTurn is fine-tuned with lightweight heads to predict six self-supervised signals per channel: EOT, HOLD, BOT, BC, VAD, and FVAD. The signal definitions are operational. EOT is a speech offset where the other speaker takes floor within 4 s. HOLD covers all other speech offsets. BOT is a speech onset at least 1 s following the other speaker. BC is an isolated utterance of at most 1 s with at least 1 s silence before and after. VAD is binary voice activity per frame. FVAD is mean voice activity over future horizons [0–240 ms], [240–480 ms], [480–960 ms], and [960 ms–2 s]. By construction, EOT and HOLD are complementary at every speech offset (Rajaa, 9 Mar 2026).

The fine-tuning losses follow the sparsity structure of the labels. Focal loss is used for sparse event signals such as EOT, HOLD, BOT, and BC, while binary cross-entropy is used for dense signals such as VAD and FVAD. Labels are temporally smoothed with an asymmetric Gaussian with $\sigma = 3$ frames before each event and $\sigma = 1$ frame after, encouraging anticipation up to 240 ms early (Rajaa, 9 Mar 2026).

The architecture uses Mimi neural codec representations, channel-specific MLPs for each speaker, concatenation of the two channel representations, and a Qwen2.5-0.5B backbone. It has 12 lightweight classification heads in total, six per channel; sparse signals use two-layer MLP heads with GELU and dropout, while dense signals use linear heads. Streaming inference uses a 240 ms stride and KV caching. Reported latency is about 78 ms on CPU for the unquantized base model and about 27 ms on A100 GPU (Rajaa, 9 Mar 2026).

3. Agent-action mapping and benchmark results

The signal layer is mapped to five interpretable agent actions: ST (Start-talking), CL (Continue-listening), SL (Start-listening), CT (Continue-talking), and BC (Backchannel). These actions go beyond endpoint detection because they encode overlap management and backchannel behavior in addition to floor transfer. The paper describes both heuristic threshold mappings, such as

$$\text{EOT}_u > 0.5 \wedge \text{BOT}_a > 0.5 \Rightarrow \text{ST},$$

and a multinomial logistic-regression probe over signal probabilities (Rajaa, 9 Mar 2026).

On Switchboard and otoSpeech, DualTurn outperforms VAP on all five agent-action classes. It also performs strongly on word-level turn prediction against a much larger audio-text baseline. The most salient reported numbers are summarized below.

Benchmark	DualTurn	Comparator
Switchboard agent-action prediction, wF1	0.633	VAP (LR-6): 0.389
otoSpeech agent-action prediction, wF1	0.707	VAP (LR-6): 0.461
Switchboard word-level turn prediction, average AUC	0.930 (heuristic), 0.963 (LR probe)	RP+HuBERT+history: 0.880

Backchannel prediction is a particularly strong differentiator. On Switchboard, VAP has BC F1 = 0.000, whereas DualTurn reaches BC F1 = 0.349; on otoSpeech, DualTurn reaches BC F1 = 0.512. The paper notes that chance F1 for BC is about 0.080, so the reported BC values are materially above chance (Rajaa, 9 Mar 2026).

DualTurn also anticipates boundaries earlier than VAP. Its median reaction time is reported as −360 ms relative to turn end, compared with −140 ms for VAP, so it anticipates boundaries about 220 ms earlier. Misfires are reduced: ST-for-CL confusions drop from 27.4% to 22.4%, ST F1 improves to 0.829 versus 0.808 for VAP, and interruptions fall by 5 percentage points (Rajaa, 9 Mar 2026).

Ablation results identify the representation-learning stage as the main source of gain. Removing Stage 1 pretraining causes large drops in weighted F1 and BC F1. Continuous Mimi embeddings outperform discrete codebook indices, with wF1 0.633 versus 0.602 and BC F1 0.349 versus 0.072. Keeping the auxiliary generative loss during Stage 2 hurts sparse event learning, with BC F1 dropping from 0.349 to 0.077. Adding a text output objective in Stage 1 also reduces performance, with wF1 0.633 versus 0.605 and BC F1 0.349 versus 0.085. This suggests that audio-only generative pretraining is more effective for turn-taking than text-aligned pretraining in this setting (Rajaa, 9 Mar 2026).

4. Multiparty and role-conditioned extensions

Recent multiparty work develops related decompositions of the turn-taking problem. In “Fast When, Careful Who,” the core proposal is a two-stage audio-only pipeline for multiparty turn-taking on VoxConverse: a fast trigger scans the audio and proposes candidate end-of-turn times, and a lightweight verifier runs only at those times to decide Hold or Shift and support next-speaker prediction (Patamia et al., 15 Jun 2026). The trigger uses a pretrained WavLM encoder with a 1.0 s window and 0.1 s hop, while the verifier uses pretrained ECAPA-TDNN speaker embeddings with 1.5 s pre- and post-context. On full multiparty SHIFT detection, the reported F1 scores are 0.233 for the baseline, 0.340 for the baseline with diffusion augmentation, 0.528 for the proposed method, and 0.635 for the proposed method with diffusion augmentation; the corresponding MAE improves from 0.189 ± 0.134 to 0.131 ± 0.118 with diffusion augmentation. In a controlled dyadic top-2 projection, the proposed method reaches F1 = 0.622 and BAcc = 0.735, compared with VAP at F1 = 0.449 and BAcc = 0.609 (Patamia et al., 15 Jun 2026).

A distinct extension appears in “Adaptive Turn-Taking for Real-time Multi-Party Voice Agents,” which proposes ModeratorLM, a role-conditioned, chunk-wise streaming speech LLM for multi-party conversations (Mitra et al., 11 Jun 2026). The model first predicts whether it should take the floor at all, conditioned on an explicitly assigned role, and only then produces a response if it takes the floor. A reasoning-augmented variant, ModeratorLM-Think, incorporates chain-of-thought reasoning over conversational context and the assigned role. On NOTSOFAR-1, ModeratorLM-Think reports Precision 0.81, Recall 0.74, F1 0.76, Accuracy 0.86, FP 0.01, and Reactive miss rate 0.02. On RolePlayConv, it reports Precision 0.79, Recall 0.82, F1 0.79, Accuracy 0.91, FP 0.03, and Reactive miss rate 0.03. The paper summarizes the gains as precision improvement over 40%, recall improvement more than 70%, and substantially reduced false-positive interruptions relative to non-role-conditioned baselines (Mitra et al., 11 Jun 2026).

Taken together, these systems indicate that turn-taking research is moving from binary endpointing toward structured decision layers: anticipating boundary timing, verifying whether the floor is actually transferring, determining who should speak next, and conditioning intervention policy on role. This suggests a broader family of models in which turn-taking is treated as a control problem rather than a pause detector.

5. Turn-level control in dialogue safety and tool-calling agents

Turn-level reasoning has also become central in text-based dialogue safety. “One Turn Too Late” studies hidden malicious intent in multi-turn dialogue and defines the problem as detecting the earliest turn at which delivering the candidate response would make the accumulated interaction sufficient to enable harmful action (Shen et al., 7 May 2026). The interaction state at turn $t$ is

$$x_t = \left((q_1, r_1), \ldots, (q_{t-1}, r_{t-1}), q_t, \tilde r_t\right),$$

and the harmful closure turn is

$$t^*(\tau, g) = \min\bigl(\{t \in \{1, \ldots, T\} : \mathrm{Suff}(x_t,g)=1\} \cup \{\infty\}\bigr).$$

To train and evaluate this objective, the paper introduces MTID, which contains 8,000 harmful dialogues and 8,000 benign dialogues generated from 400 harmful and 400 benign seeds across Chemistry and Cybersecurity, with annotations of the first harm-enabling turn. The resulting response-aware monitor, TurnGate, improves Harmful Score $\phi_2$ from 0.479 to 0.602 and F1 $\phi_2$ from 0.610 to 0.699 on MTID test data, while exact-turn accuracy improves from 34.3% to 41.4% and miss rate falls from 37.9% to 17.7% (Shen et al., 7 May 2026).

A related turn-level perspective appears in multi-turn tool-calling. “Self-Evolution for Multi-Turn Tool-Calling Agents via Divergence-Point Preference Learning” studies tau2-bench, a dual-control setting in which an agent alternates between dialogue turns and tool execution turns while conditioning on history $h_t$ and ToolGraph control context $\sigma = 3$0 through $\sigma = 3$1 (Tang, 22 Jun 2026). ToolGraph raises weighted average reward from 0.304 to 0.338, and ToolGraph+DPO reaches 0.355 across 375 tasks, corresponding to +11.2% relative improvement for ToolGraph alone and +16.8% over the baseline for ToolGraph+DPO. The preference-learning stage uses 161 preference pairs extracted at divergence points between successful and failed trajectories, filtered with action-correctness annotations. Fine-grained diagnostics show that roughly half of telecom trajectories exhaust the 50-step budget before action execution, and that positive chosen reward is the most useful checkpoint signal across the 16 evaluated DPO configurations (Tang, 22 Jun 2026).

These papers do not use DualTurn as the name of the method, but they adopt the same turn-level granularity that distinguishes closure points, intervention timing, and state-dependent action selection from coarser single-turn or trajectory-level judgments.

6. Distinct usage outside conversational AI

The label DualTurn is also used in a different field by “Coherent Turning Behaviors Revealed Across Adherent Cells,” where it names a runner-rotor picture of adherent-cell migration (Zhang et al., 26 Mar 2025). In that formulation, cells alternate between two kinematic states: a Run state, which is persistent in direction with strongly suppressed angular change, and a Turn state, in which the cell’s direction changes approximately at a constant angular velocity with random sign. The proposed turning law is

$\sigma = 3$2

in contrast to rotational-diffusion-based models of the form $\sigma = 3$3. A turn is detected by thresholding angular velocity,

$\sigma = 3$4

Turn and run durations are exponentially distributed, consistent with a two-state Markov process, with $\sigma = 3$5 min and $\sigma = 3$6 min for MCF-10A on collagen, implying that cells spend about 80% of their time in the persistent run state (Zhang et al., 26 Mar 2025).

Across cell types and morphologies, turns behave as noisy rotors. The reported turning rates are approximately 0.160 rad/min for MCF-10A on collagen, 0.090 rad/min for MCF-10A on PDMS, and 0.082 rad/min for NIH-3T3 on PDMS, whereas the effective angular rate in runs is about 0.006–0.007 rad/min. The angular autocorrelation during turns is fit by

$\sigma = 3$7

with reported examples such as $\sigma = 3$8 and $\sigma = 3$9 for MCF-10A on collagen. In mechanotaxis experiments using cyclic substrate pulling with a micropipette about $\sigma = 1$0 to the right of the cell, about 86% of turns in arrived cells helped move the cell toward the pipette. The interpretation is that tactic motion can be achieved by biasing the sign of turns toward the cue without changing the constant-rate turning mechanism itself (Zhang et al., 26 Mar 2025).

A separate, unrelated astrophysical literature studies double or extended main-sequence turnoffs in intermediate-age massive star clusters rather than a method called DualTurn. There, rotation can lead to a double or extended MSTO for clusters with ages between about 0.8 and 2.2 Gyr, with a typical apparent age spread of about 100–200 Myr and in some weaker-mixing cases about 400 Myr (Yang et al., 2013). This suggests that the string “DualTurn” can refer either to a named methodological framework or to unrelated “dual turnoff” terminology, depending on domain.

In aggregate, the term denotes a family resemblance rather than a single universal theory: a decomposition of behavior into complementary turn states, turn stages, or turn-level decisions. In spoken dialogue, the best-defined usage is the dual-channel generative turn-taking model that continuously predicts interpretable signals and agent actions (Rajaa, 9 Mar 2026). In adjacent literatures, the same label or closely related formulations describe deterministic rotor-like turning in cells (Zhang et al., 26 Mar 2025), candidate-trigger plus verifier pipelines for multiparty speech (Patamia et al., 15 Jun 2026), and turn-level intervention in multi-turn dialogue safety and tool orchestration (Shen et al., 7 May 2026).