---
title: 'Full-Duplex-Bench: Real-Time Dialogue Benchmark'
url: https://www.emergentmind.com/topics/full-duplex-bench
type: topic
---

# Full-Duplex-Bench: Real-Time Dialogue Benchmark

Full-Duplex-Bench is an open, systematic benchmarking framework for quantifying the interactive capabilities of full-duplex spoken dialogue models (SDMs) that both listen and speak simultaneously under realistic, real-time conditions. Unlike traditional half-duplex evaluation, which assesses models in rigid, sequential turn-taking, Full-Duplex-Bench specifically targets the nuanced, overlapping, and interruptive phenomena characteristic of natural human conversation. Across its versions and extensions, Full-Duplex-Bench has established itself as the primary yardstick for evaluating fundamental dialogue behaviors, robustness to conversational disfluency, tool-use performance, and advanced scenario management in both open-source and commercial full-duplex SDMs [2503.04721] [2507.23159] [2510.07838] [2604.04847].

## 1. Background and Motivation

Historically, nearly all spoken dialogue models operated in half-duplex: the user speaks, the agent waits, listeners alternate in strictly serialized turns, and models are evaluated on content accuracy or coarse temporal statistics. This structure inherently misses critical real-time behaviors found in human-human dialogue, such as:

- **Overlapping speech:** Natural conversations feature frequent overlaps, pauses, and backchannel feedback.
- **Barge-in and repair:** Effective agents must correctly handle user interruptions, revisions, and rapid turn-exchanges.
- **Backchanneling and engagement:** Listeners signal engagement non-intrusively, an essential dynamic for conversational naturalness.

Full-duplex SDMs, exemplified by systems like GPT-4o voice mode, Moshi, and Freeze-Omni, allow simultaneous listening and speaking but introduce new failure modes: inappropriate interruptions, ignored user cues, and incorrect response timing. There was no reproducible, open-standard method for evaluating these capabilities until the introduction of Full-Duplex-Bench [2503.04721]. Subsequent iterations have expanded the evaluation axis to multi-role, multi-task, and real-world tool-use settings [2507.23159] [2510.07838] [2604.04847].

## 2. Core Behavioral Dimensions and Metric Formalizations

The original Full-Duplex-Bench formalizes full-duplex interaction along four primary behavioral axes [2503.04721]:

- **Pause Handling:** Determines if a model misclassifies intra-turn user silences (e.g., hesitations) as turn boundaries. The metric is the Takeover Rate (TOR), where TOR$\to$0 is ideal.
  
  $$
  \mathrm{TOR} = \frac{1}{N}\sum_{i=1}^N \mathrm{TO}_i, \quad
  \mathrm{TO}_i = \begin{cases} 1 & \text{model takes over} \\ 0 & \text{otherwise} \end{cases}
  $$
  
- **Backchanneling:** Quantifies the model’s ability to offer brief, well-timed listener feedback. Metrics: TOR (for deferring turns), Backchannel Frequency (per second of user speech), and Jensen–Shannon Divergence (JSD) for alignment with human backchannel timing distributions.

- **Smooth Turn-Taking:** Measures response latency and temporal coordination after user turn-ends:

  $$
  L = \frac{1}{K} \sum_{i:\mathrm{TO}_i=1} (t^{(m)}_{\mathrm{start},i} - t^{(u)}_{\mathrm{end},i})
  $$

- **User Interruption Management:** Assesses model reaction to mid-utterance user barge-ins—detecting, yielding, and responding to new intent. Metrics include TOR (should be 1 in interruption condition), GPT-4o semantic relevance score (0–5), and response latency post-interruption.

Later versions, such as Full-Duplex-Bench v1.5, introduce categorical dialogue behaviors (Respond, Resume, Uncertain, Unknown), stop/response latency, prosodic adaptation (pitch, intensity, speaking rate), and MOS prediction for segmental speech quality [2507.23159].

## 3. Data, Pipelines, and Experimental Protocols

Full-Duplex-Bench assembles carefully partitioned corpora and automated measurement scripts for reproducible experimentation:

- **Pause/Turn-Taking Data:** Extracted from large-scale, spontaneous two-channel English dialog datasets (e.g., CANDOR), filtered by intra-turn pause or inter-turn gap duration for targeting the correct phenomena [2503.04721].
- **Backchanneling:** Uses annotated corpora with human backchannel timing, segmented and normalized to provide reference distributions [2503.04721].
- **Synthetic Interruptions:** Scenarios are generated via prompt-based synthesis (e.g., GPT-4o, ChatTTS), spanning a range of interruption types and timing conditions [2503.04721] [2507.23159].
- **Streaming Evaluation:** All model outputs and user streams are resampled, transcribed (e.g., with CrispyWhisper or Parakeet-TDT), then segmented via VAD for precise timestamp annotation. Metric computation is fully automated for batch evaluation with user-configurable thresholds and scenario selection.

The benchmark includes baseline results on leading open-source systems (dGSLM, Moshi, Freeze-Omni), systematically reported across all metrics [2503.04721].

## 4. Key Results and Model Comparison

Experimental evaluations reveal significant differences in how modern SDMs negotiate full-duplex settings. Comparative findings include [2503.04721] [2507.23159] [2604.04847]:

| Dimension                       | dGSLM (TOR) | Moshi (TOR) | Freeze-Omni (TOR) |
|----------------------------------|-------------|-------------|-------------------|
| Pause Handling (synthetic)       | 0.949       | 1.000       | 0.672             |
| Pause Handling (Candor)          | 0.953       | 0.989       | 0.287             |
| Backchannel (TOR / Freq / JSD)   | 0.782/0.013/0.950 | 1.000/0.005/0.977 | 0.782/0.002/0.984 |
| Smooth Turn-Taking (TOR/Latency) | 0.989/0.572 | 0.996/0.112 | 0.369/1.168       |
| User Interruption (TOR/Score/Lat)| 0.895/0.201/3.972 | 1.000/0.765/0.037 | 0.795/3.371/1.200 |

- MOSHI replies with minimal latency but frequently interrupts or under-backchannels.
- Freeze-Omni’s explicit state-control yields the lowest TOR in pause handling but at the expense of increased response latency.
- dGSLM produces the most human-like backchannel timing (lowest JSD) but exhibits over-eager turn responses.

Benchmarks on advanced scenario categories—overlap, background distractors, corrections, and tool chaining—consistently show that interruption management and self-correction remain open research challenges for all architectures [2507.23159] [2510.07838] [2604.04847].

## 5. Extensibility, Language Coverage, and Successors

Full-Duplex-Bench is designed for plug-and-play extensibility:

- **New audio assets:** Support for real or synthetic speech; audio-asset interchangeability with explicit onset/offset annotation.
- **Scenario injection:** Arbitrary overlap types, environmental conditions (side-conversations, ambient noise), and complex barge-in profiles.
- **Language extension:** Localization through scenario/script translation and normalization of the segmentation/VAD pipeline.

All code, scenario definitions, and metric scripts are open-sourced, facilitating easy adaptation and extension to novel research questions [2503.04721] [2507.23159].

- **Full-Duplex-Bench v1.5:** Systematic analysis of overlap handling introduces repair-first versus continuity-first agent strategies, scenario-specific performance trends, and prosodic adaptation statistics [2507.23159].
- **FDB-v2 and v3:** Integrate automated examiners (streaming speech-LLMs), multi-turn stateful tasks (entity tracking, corrections, safety), disfluency-annotated real human audio, and multi-step tool-use chains [2510.07838] [2604.04847].

## 6. Applications and Limitations

**Applications:**

- Rigorous, reproducible benchmark for emerging speech-driven LMs in academia and industry.
- Diagnostic tool for pinpointing conversational breakdowns in full-duplex agent architectures—enabling ablations and architectural comparisons.
- Essential for measuring role-conditional behavior, naturalness (DMOS), timing fluidity, memory in multi-turn, and robustness to disfluent, noisy, or adversarial input.

**Limitations:**

- Initial releases limited to English; cross-cultural turn-taking and backchannel cues are not covered [2503.04721].
- Heuristic detection for backchannels; paralinguistic (laughter, sighs) and non-lexical phenomena not comprehensively measured.
- Commercial, closed-source systems must be evaluated via API wrappers; not locally reproducible.
- Subjective scoring uses automated LLMs (e.g., GPT-4o) in lieu of crowd-sourced human ratings; future work could improve external validity [2503.04721] [2510.07838] [2604.04847].

## 7. Impact and Outlook

Full-Duplex-Bench has defined core quantitative standards for turn-taking, interruption, backchanneling, and tool-use in real-time voice agents. Extensions support advanced conversational setting: role-specific interactions, multi-turn flows, and naturalistic disfluency. The benchmark’s automation, open-source release, and modular architecture have enabled rapid progress and transparent comparison across leading full-duplex agent platforms [2604.04847].

Further advances are directed at expanding language and cultural coverage, integrating richer paralinguistic markers, combining LLM and human evaluation for subjective metrics, and supporting a broader array of deployment scenarios (e.g., multi-party, multi-modal, cross-lingual). Full-Duplex-Bench remains the reference test suite for spoken dialogue models seeking human-parity in conversational dynamics.

Source: https://www.emergentmind.com/topics/full-duplex-bench