- The paper introduces DuplexChat-Pipe, a scalable, open-source pipeline for constructing large-scale, speaker-separated dialogue corpora from podcast feeds.
- It employs advanced diarization and diffusion-based speech separation techniques, achieving superior audio quality and realistic turn-taking compared to Fisher benchmarks.
- The extensive DuplexChat corpus enables robust training for spoken dialogue language models and supports future research on natural, cross-lingual conversational AI.
DuplexChat: Scaling Full-Duplex Speaker-Separated Dialogue for Spoken Dialogue Language Modeling
Motivation and Background
Spoken Dialogue LLMs (SDLMs) represent a significant departure from traditional, cascaded spoken dialogue systems by enabling end-to-end modeling of human-like conversational dynamics—including rapid turn-taking, backchannels, and overlapping speech. Realizing these capabilities at scale necessitates large corpora of full-duplex, speaker-separated conversational audio, directly analogous to the scaling dependencies observed in text-based LMs [kaplan2020scalinglawsneurallanguage]. However, the scarcity of large-scale, real conversational datasets with separate channels for each speaker—typically restricted to telephone-based datasets such as CallHome and Fisher [ldc2008_cabank_callhome_english, cieri-etal-2004-fisher]—remains a critical bottleneck for SDLM advancement. Web-scale audio corpora, while abundant (e.g., GigaSpeech [chen21o_interspeech], Spotify Podcast Dataset [clifton-etal-2020-100000]), almost universally present speech in a monaural, mixed-channel format, fundamentally limiting their utility for full-duplex modeling.
DuplexChat-Pipe: Pipeline Architecture and Methodology
DuplexChat-Pipe is introduced as an open-source, re-runnable pipeline for constructing large-scale, speaker-separated full-duplex dialogue corpora from publicly available podcast feeds. The pipeline comprises four primary stages:
- Feed Collection and Language Filtering: Using PodcastIndex, language-matched RSS feeds are identified and deduplicated, forming the initial set of candidate podcast sources.
- Audio Retrieval and Cleaning: Metadata-based filtering removes non-dialogue and music content, followed by download, resampling to 16 kHz, and duration-based filtering to eliminate excessively long, likely off-target episodes.
- Diarization-Based Dialogue Segmentation: Leveraging pyannote-based speaker diarization, episodes are segmented into two-speaker interactions, with stringent rejection criteria for monologues, non-dialogue, or clips failing minimum duration and speaker alternation thresholds.
- Speech Separation and Restoration: Each two-speaker clip is processed with DialogueSidon [dialoguesidon], a diffusion-based model offering joint speech separation, denoising, and dereverberation. Output comprises stereo representations assigning each separated speaker to a distinct channel.
The ethics guardrail is notable: the released dataset comprises only metadata and segment durations, requiring users to reconstruct the underlying audio themselves, thereby sidestepping direct copyright risks.
DuplexChat Corpus: Scale and Quantitative Properties
DuplexChat is the product of applying DuplexChat-Pipe at web scale, yielding unparalleled volume and coverage: 282,634 hours of English and 132,723 hours of Japanese two-speaker, full-duplex dialogues, extracted from over 13,000 (English) and 9,000 (Japanese) unique podcast feeds. The English side leverages merely ~2% of the available English feeds, with growth potential exceeding current scale by at least an order of magnitude.
Audio Quality and Separation Benchmarking
The acoustic and separation quality of DuplexChat was benchmarked against Fisher—widely regarded as a gold-standard dataset for SDLM training. Evaluation utilized DNSMOS, SQ-STOI, SQ-PESQ (via SQUIM), as well as intra-track consistency (ITC) and inter-track distinctiveness (ITD) metrics for separation fidelity.
- English DuplexChat demonstrates superior cleanliness (DNSMOS=2.92, SQ-STOI=0.96, SQ-PESQ=3.23) and separation (ITC=0.383, ITD=0.880) relative to Fisher.
- Japanese DuplexChat achieves even higher intelligibility (DNSMOS=3.11, SQ-STOI=0.98, SQ-PESQ=3.30), although ITC is slightly lower, attributed to the separation model's limited exposure to Japanese in its training data.
These results support the suitability of DuplexChat as a training resource for high-fidelity, speaker-consistent SDLMs.
Turn-Taking and Interactional Dynamics
Statistical analysis of turn-taking phenomena reflects authentic conversational dynamics, with language-dependent sociolinguistic features:
- Japanese dialogues exhibit higher turn-exchange rates (10.5/min vs 6.2/min in English), more backchannels, and increased simultaneous speech, consistent with established cross-linguistic analyses [doi:10.1073/pnas.0903616106, liberman2007towards].
- Overlapping transitions account for nearly 50% of all turn-taking events in both languages, indicating rich modeling targets for SDLMs that aim to capture interactivity beyond simple turn-based structures.
Collectively, these characteristics position DuplexChat as a singular resource for realistically modeling the temporal and interactional patterns inherent in natural speech.
Practical and Theoretical Implications
The pipeline's scalability—demonstrated by using diverse, continually updated internet sources—removes the largest obstacle for scaling SDLM training: data acquisition. DuplexChat's magnitude exceeds all previously published open spoken dialogue corpora, repositioning the community to match the scaling laws that have driven recent breakthroughs in text modeling [kaplan2020scalinglawsneurallanguage].
On a practical axis, DuplexChat reduces the barrier for training robust, cross-lingual SDLMs capable of supporting end-to-end, full-duplex voice interfaces. Its open-source pipeline paradigm also enables reproducibility, continuous improvement via new separation/restoration models, and straightforward extension to additional languages and modalities as diarization and source separation technology advances.
On a theoretical level, the resource enables systematic study of conversational timing phenomena, overlap resolution, and interaction management in spoken LMs. Furthermore, large-scale full-duplex corpora will likely catalyze new modeling architectures and training objectives oriented around reaction time, synchrony, and multi-party dialogue, expanding the design space for conversational AI.
Future Prospects
The authors highlight the need for future research to assess the impact of DuplexChat on SDLM training efficacy at scale, specifically evaluating its influence on real-time responsiveness, naturalness, and robustness in dialogue understanding and generation. The pipeline's extensibility to even larger, more diverse language pools suggests a path towards genuinely universal voice-based models.
There is scope for improved separation in underrepresented languages by co-training new restoration/separation models on the rapidly growing DuplexChat outputs, closing performance gaps across linguistic domains.
Conclusion
DuplexChat and DuplexChat-Pipe constitute a substantial methodological and practical advancement for the SDLM community, addressing the acute shortage of web-scale, speaker-separated full-duplex dialogue resources (2607.04941). The demonstrated improvements in data scale, audio quality, and natural interactional structure have wide-reaching practical implications for both foundational research and the deployment of conversational AI agents, serving as a critical enabler for scaling spoken dialogue LMs towards greater generality and naturalism.