- The paper introduces a four-tier architectural hierarchy and a five-state decision machine to diagnose full-duplex system performance.
- It presents an interaction ontology (T×I×R) that maps dialogue behaviors for guiding system design and evaluation.
- The study identifies data bottlenecks as the key constraint, advocating new synthesis and training strategies for improved cell coverage.
Authoritative Analysis of "A Survey of Full-Duplex Spoken Dialogue Systems: Architectural Hierarchy, Interaction Ontology, and Decision State Machine" (2606.19453)
Introduction
This survey systematically addresses conceptual, data, and evaluation ambiguities in the rapidly expanding field of full-duplex spoken dialogue systems (FD-SDS). The authors critique previous works for collapsing technical distinctions into a one-dimensional design axis, and subsequently develop three rigorous frameworks: (1) an L0–L3 architectural hierarchy delineating where duplex decisions are made; (2) a T×I×R interaction ontology grounding per-moment behavior; and (3) a five-state decision state machine formalizing system transition dynamics. Through these frameworks, the paper audits state-of-the-art systems and proposes novel hypotheses for the architectural and data frontiers.
Architectural Hierarchy: L0–L3 Paradigm
The survey introduces a four-tier hierarchy for locating system duplex decisions:
- L0: Module-Level — External schedulers (VAD/EoT/dialogue manager) arbitrate speaker turns independent of the LLM backbone. Exemplified by FireRedChat, FlexDuo, X-Talk, and SoulX-Duplug, these architectures persist in industrial pipelines, particularly for their robustness and low-latency properties. Modern extensions fuse acoustic and linguistic cues for enhanced turn detection (e.g., FastTurn's dual-signal fusion).
- L1: Hidden-State-Level — Duplex decisions rendered via predictors reading LLM hidden states. MinMo and Freeze-Omni fully instantiate this pattern. The ubiquitous "Thinker–Talker" decomposition found in Qwen2.5-Omni, Qwen3.5-Omni, and Step-Audio R1.1 reflects its recurrent utility, independent of duplex claims.
- L2: Token-Level — Duplex encoded into the token generation process itself. Moshi, LSLM, OmniFlatten, SyncLLM, Mini-Omni2, Fun-Audio-Chat, and Covo-Audio populate this space. Moshi, for example, utilizes parallel token streams over semantic–acoustic-fused codecs, while OmniFlatten achieves full-duplex dialogue via careful sequence design without architectural modification.
- L3: Representation-Level — Theoretical frontier: shared latent representations integrating user and assistant streams at the continuous level. No published systems exist; candidate approaches discussed include continuous-latent AR streaming and JEPA-style dual-stream prediction.
The analysis reveals that both L1 and L2 are structural attractors with practical deployments, while L0 is not simply legacy but remains a competitive design trajectory. L3 is identified as the principal architectural gap.
Interaction Ontology: T×I×R Framework
The interaction space is defined by orthogonal axes:
- T (Temporal relation): Sequential, latched, overlap, concurrent, silence.
- I (User intent): Information, backchannel, repair, floor-claim, floor-yield, disfluency, third-party.
- R (System response): Continue, stop, wait, backchannel, ignore, initiate.
Six acid-test cells are highlighted, mapping onto canonical communicative scenarios critical for robust evaluation. These include cooperative barge-in, backchannel during system speech, third-party noise rejection, and hesitation handling — elements frequently overlooked in prior benchmarks.
A full-duplex system is modeled as occupying one of five states (IDLE, LISTEN, SPEAK, WAIT, DUAL), with moment-to-moment transitions precisely triggered by the (T,I,R) ontology. Eleven transitions encompass onset, turn-handoff, and overlap scenarios; each transition is mapped to concrete behavioral tests, providing a cross-system ablation and trace inventory. This formalism encapsulates the decisive distinction between substantively and apparently full-duplex systems, e.g., causal-influence versus chunk-aligned designs.
Data Bottleneck: Training Corpus Analysis
By auditing leading FD systems, the paper substantiates a strong data-bottleneck thesis: architectural capability is necessary but insufficient — the primary constraint is the cell-coverage of training data, especially for complex interaction cells like sustained concurrent speech (T4) and third-party voice (T3·I7). Public corpora (Fisher, Switchboard, CANDOR, Easy Turn, HumDial Track2) are dominated by English, lose coverage on T4, and are orders of magnitude smaller than undisclosed industrial datasets. Synthesis pipelines, cell-driven corpus rebalancing, and RL-based objectives provide potential amelioration routes, but sustained concurrent speech remains essentially unreachable absent real data.
Evaluation: Realization Gap and Benchmark Landscape
Standard metrics (WER, MOS, BLEU, turn latency) fail to capture FD-SDS dynamics. The field has established two main benchmark families: Full-Duplex-Bench (FDB v1–v3) and FD-Bench, each with distinct scenario and metric focus. FDB v1.5 is notable for its overlap decomposition, directly assessing barge-in, backchannel, and third-party speech scenarios. The paper introduces the realization gap as a diagnostic frame: architectural capacity versus realized behavior, with empirical evidence showing that systems with identical L2 architectures can diverge dramatically in demonstrated cell coverage due to training data. Multicomponent evaluation (architecture, data profile, demonstrated behavior) is recommended for all future FD system reports.
Significant structural gaps exist in the current benchmark regime: lack of adaptive interaction protocols, insufficient language/acoustic diversity, and absence of explicit causal-influence probes, limiting the ability to distinguish substantive from apparent FD behavior.
Architectural Frontier: L3 Representation-Level Hypotheses
Three architectural directions are posited for L3 systems:
- Continuous-Latent Autoregressive Streaming: AR Transformer predicting continuous-valued latent streams, bypassing codec/tokenization bottlenecks.
- JEPA-Style Dual-Stream Prediction: Masked latent region prediction from joint context, inherently integrating overlap dynamics.
- World-Model-Conditioned Dialogue: Latent encoding controlling the perceived world state, mapping dialogue events directly into latent transitions rather than external state variables.
Obstacles include entrenched discrete-tokenizer ecosystems, unresolved real-time latency in continuous-latent models, and the absence of consensus latent formulation for dialogue. The survey identifies these as pivotal technical agendas for the field.
Implications and Future Directions
Practical Implications
For system builders, this survey provides explicit architectural, data, and evaluation recipes for full-duplex system development. The modular L0 path is affirmed for its robustness; the token-level L2 path is dominant in academia, though heterogeneity remains. Synthesis pipelines for cell augmentation, cell-driven corpus alignment, and novel training objectives are identified as high-leverage directions to bridge the public–industrial gap. Immediate practical impact centers on improved sampling for rare interaction cells and per-cell evaluation reporting.
Theoretical Implications
The formal frameworks expose new theoretical typologies for dialogue system research, demarcating empirical limits binding current designs and outlining the path toward fully integrated L3 systems. The realization gap operationalizes the coupling between architecture and data, providing a rigorous interpretive lens for future advances. The exposition of architectural attractors (L1/L2) and unserved interaction cells raises critical questions for both representation learning and corpus engineering.
Future Speculation
In the near term, the field will likely see expanded synthesis techniques for rare cell coverage, the proliferation of adaptive and causal-influence evaluation protocols, and continued modular reinvention at the L0 layer. Longer-term progress toward L3 architectures—continuous-latent, JEPA-inspired, world-model-driven—will demand new mathematical formalisms and streaming generative frameworks. This transition will also necessitate extending current ontology and state machine paradigms, particularly as novel interaction types become salient.
Conclusion
The paper delivers a rigorous taxonomical, data-driven, and evaluation-grounded framework for analyzing and designing full-duplex spoken dialogue systems. It systematically replaces ambiguous "full-duplex" claims with structured diagnostic profiles and exposes the cross-system gap as fundamentally coupled to training data and evaluation protocols. The survey's conceptual, empirical, and forward-looking contributions will serve as a foundation for both industrial and academic development of next-generation full-duplex systems. Future research trajectories are explicitly delineated, notably in architectural modeling and corpus design, with the L3 paradigm as the principal open frontier.