LEMAS-Dataset: Multilingual Speech Corpus

• LEMAS-Dataset is a large-scale multilingual open speech corpus that includes 150K hours of audio across 10 major languages with rigorous word-level timestamps.
• The dataset is built using an efficient pipeline combining multilingual forced alignment, confidence-based filtering, and schema unification from diverse public sources.
• It supports cutting-edge research in generative speech technologies by underpinning systems like LEMAS-TTS for synthesis and LEMAS-Edit for precise speech editing.

Searching arXiv for the primary LEMAS paper and later usage references.

LEMAS-Dataset is the dataset component of “LEMAS: A 150K-Hour Large-scale Extensible Multilingual Audio Suite with Generative Speech Models” and is presented as an open multilingual speech corpus for generative speech research, especially multilingual text-to-speech, speech editing, and prompt-based speech generation. The paper describes it, to the authors’ knowledge, as the largest open-source multilingual speech corpus with rigorous word-level timestamps, covering over 150,000 hours across 10 major languages and constructed through an efficient data processing pipeline that combines multilingual forced alignment, confidence-based filtering, and schema unification across heterogeneous public corpora (Zhao et al., 4 Jan 2026).

1. Position within multilingual generative speech research

LEMAS-Dataset is framed as infrastructure rather than as a narrow benchmark. Its intended role is to support large-scale multilingual generative speech modeling, with the paper validating that claim through two downstream systems: LEMAS-TTS, a non-autoregressive flow-matching multilingual TTS model, and LEMAS-Edit, an autoregressive speech editing model that relies directly on precise word-level alignments. The dataset is therefore positioned not only for multilingual synthesis, but also for tasks that require fine-grained temporal grounding, including duration prediction, prompt construction, and localized speech editing (Zhao et al., 4 Jan 2026).

The paper’s motivation is explicitly data-centric. It argues that prior multilingual speech resources are often monolingual, smaller in scale, domain-limited, weakly filtered, or missing reliable fine-grained timestamps. In that framing, LEMAS-Dataset is defined by the conjunction of four properties: large scale, 10-language coverage, open availability, and word-level temporal supervision. A plausible implication is that the dataset is intended as a foundation-style corpus for multilingual generative models rather than as a task-specific benchmark.

2. Data sources and construction pipeline

LEMAS-Dataset is assembled from publicly available corpora rather than from new recordings. The raw sources listed in the paper are GigaSpeech, GigaSpeech2, WenetSpeech4TTS, Emilia, MLS, multilingual TEDx, Alcaim, Golos, and YODAS. These sources span audiobooks, podcasts, and in-the-wild conversational speech, and the comparison table characterizes the overall source mix as both Audiobook and In-the-wild (Zhao et al., 4 Jan 2026).

The processing pipeline has six explicit stages. Audio-text pairs are first ingested and normalized into a unified representation. Transcripts are then subjected to language-specific text normalization and romanization, for example through Uroman, so that diverse scripts can be handled in a shared Latin-space representation. Multilingual forced alignment is then performed with the Multilingual MMS Forced Aligner from torchaudio, which is based on a wav2vec-based CTC alignment model. This stage produces word-level boundaries and token-level posterior probabilities, and those posteriors are used to derive word confidence and utterance-level average confidence (Zhao et al., 4 Jan 2026).

Filtering is a central design component. The paper states that the retained clips must satisfy source-specific alignment-confidence thresholds with $x \in [0.2, 0.5]$, duration limits between 0.5 s and 30 s, silence or unaligned-region constraints below 4 s, language validation against the 10 target languages, and character validation that removes unsupported scripts, emojis, and excessive non-linguistic symbols. It also imposes a language-specific speech-rate constraint of the form

$$\text{len(text)} / \text{duration} \in [\text{min\_ratio}_L, \text{max\_ratio}_L].$$

The dataset is finally stored in a JSON-based format, with audio converted to MP3 while preserving original sampling rates (Zhao et al., 4 Jan 2026).

A notable negative fact is that the paper explicitly says no explicit speech enhancement or background noise removal is applied during preprocessing. Deduplication, VAD, speaker verification, and speaker clustering are not described as dataset-construction stages.

3. Scale, language coverage, and released schema

The training split contains 150.14k h, 133.71M utterances, and 1,543.34M words, with an overall average duration of 4.18 s and average words per utterance of 12.02. The 10 languages are Chinese, English, Russian, Spanish, Indonesian, German, Portuguese, Vietnamese, French, and Italian. The paper states that the dataset is explicitly curated to balance duration across languages, although Chinese and English remain substantially larger than the smallest language subsets (Zhao et al., 4 Jan 2026).

Language	Total Duration	Utterances
zh	32.92k h	17.78M
en	25.35k h	9.52M
ru	22.92k h	27.47M
es	21.22k h	26.41M
id	11.25k h	11.66M
de	9.84k h	11.00M
pt	7.38k h	8.68M
vi	6.60k h	6.43M
fr	6.54k h	7.56M
it	6.12k h	7.21M

The released unified JSON representation is described as containing a unique sample key, audio file path, original transcript, normalized transcript, word-level timestamps, and confidence scores associated with word-level timestamps. The paper also makes clear that language identity is operationally known throughout processing, although it does not explicitly enumerate language ID as a release field. By contrast, it does not state that the released dataset includes phoneme labels, phone-level alignments, speaker labels, demographics, or speaker embeddings (Zhao et al., 4 Jan 2026).

The evaluation split is curated much more aggressively. It contains 500 utterances per language, for 5,000 total utterances, 470.73 minutes in total, average duration 5.65 s, and 78,722 total words. Selection criteria include average word-level alignment score $> 0.9$, word count $> 5$, duration between 3 and 15 seconds, sentence-end silence trimmed to $\le 0.2$ s, and sampling near each language’s mean character-to-duration ratio; approximately 20% of the evaluation set is manually reviewed to verify that the samples are natural speech (Zhao et al., 4 Jan 2026).

4. Word-level timestamps as the defining annotation

The distinctive annotation of LEMAS-Dataset is the combination of word-level timestamps and confidence scores. The paper treats these not as an auxiliary convenience but as the defining property that differentiates LEMAS-Dataset from most large multilingual speech corpora. In the comparison table, several large corpora—such as Libri-Light, GigaSpeech, GigaSpeech2, MLS, Emilia, and Voxbox—are listed as lacking timestamps, whereas LEMAS-Dataset is reported as providing them (Zhao et al., 4 Jan 2026).

These timestamps serve multiple roles. First, they support confidence-aware filtering, since alignment posterior probabilities can be summarized into utterance-level reliability scores. Second, they make it possible to construct duration-aware prompts and insert pause tags in downstream TTS models. Third, and most directly, they enable word-level speech editing: the paper states that LEMAS-Edit exploits precise word-level alignments to construct training masks and to perform localized infilling with smooth acoustic boundaries (Zhao et al., 4 Jan 2026).

The paper does not provide a standalone manual timestamp-accuracy study or a direct alignment error table. Instead, it validates timestamp usefulness indirectly through downstream performance, especially editing quality and multilingual TTS results. This suggests that the annotation philosophy is pragmatic: timestamps are justified operationally by what they enable, rather than by an isolated alignment benchmark.

5. Validation through LEMAS-TTS and LEMAS-Edit

The dataset is validated through two generative paradigms deliberately chosen to stress different aspects of the corpus. LEMAS-TTS is built on F5-TTS and uses a non-autoregressive flow-matching architecture with a Diffusion Transformer (DiT) backbone. It exploits dataset scale and multilinguality for zero-shot multilingual synthesis, while also using timestamp-derived pause tags and auxiliary objectives such as CTC loss and accent-adversarial training. LEMAS-Edit extends VoiceCraft into a multilingual editing framework, formulating speech editing as masked token infilling and relying directly on aligned word boundaries to define edit spans (Zhao et al., 4 Jan 2026).

The TTS benchmark is multilingual and cross-lingual in construction. The paper reports that LEMAS-TTS consistently outperforms OpenAudio-S1-mini on average, improving WER from 12.27 to 6.39 when prosody is included, while increasing SIM from 0.480 to 0.539. Reported examples include de: 3.95 → 0.44 WER, en: 4.98 → 1.07, id: 32.77 → 6.38, and zh: 9.59 → 3.00 (Zhao et al., 4 Jan 2026).

For editing, the paper uses 20 utterances sampled from the LEMAS-Dataset evaluation set across 7 languages, with edits created by replacing one word or phrase and judged by 6 human listeners in an A/B preference test. The reported outcome is that listener judgments are balanced between LEMAS-TTS and LEMAS-Edit, which the paper interprets as evidence that LEMAS can function as a robust multilingual editing benchmark across architectures (Zhao et al., 4 Jan 2026).

A later paper, “X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning”, treats LEMAS-TTS as a substantive multilingual baseline and states that LEMAS is included as a source of Russian training data. Appendix C of that paper describes the compared LEMAS-TTS system as a flow-matching based non-autoregressive model trained on 150K hours of MMS force-aligned data, using the GRL version with accent-adversarial training and CTC loss (Xu et al., 7 May 2026). This later usage indicates that LEMAS-Dataset has already begun to function as a reusable multilingual speech resource beyond its original paper.

6. Relation to prior corpora, strengths, and limitations

The paper situates LEMAS-Dataset against several prominent speech corpora. In the comparison table, Libri-Light is listed at 60k h and 1 language without timestamps; GigaSpeech at 10k h, 1 language, no timestamps; GigaSpeech2 at 28k h, 3 languages, no timestamps; WenetSpeech4TTS at 13k h, 1 language, with timestamps; MLS at 51k h, 8 languages, no timestamps; Emilia at 102k h, 6 languages, no timestamps; Voxbox at 103k h, 2 languages, no timestamps; and LEMAS-Dataset at 150k h, 10 languages, with timestamps (Zhao et al., 4 Jan 2026).

On that basis, LEMAS-Dataset’s principal strengths are explicit in the paper: scale, 10-language multilingual coverage, word-level timestamps, confidence scores, mixed-domain audio, open release, and an extensible pipeline that avoids lexicon-heavy language-specific alignment machinery. The paper repeatedly presents these features as jointly significant, since prior resources generally provide only subsets of them (Zhao et al., 4 Jan 2026).

The limitations are equally important. The paper does not report the number of speakers, gender distribution, age, accent, or other demographic statistics. It does not provide a dedicated bias audit, a large-scale manual timestamp verification study, or detailed release-schema documentation beyond the JSON fields listed in prose. It also notes, implicitly or explicitly, that Chinese and English remain larger, that source composition is uneven across languages, that no denoising is applied, and that alignment quality is filtered rather than manually verified at scale. License terms are also not stated in the paper (Zhao et al., 4 Jan 2026).

These omissions matter for interpretation. They do not diminish the dataset’s stated contribution, but they delimit what can be established from the paper itself. What is firmly documented is that LEMAS-Dataset is a very large, open multilingual corpus with word-level timestamps and confidence scores, designed to support multilingual generative speech systems and already validated through both synthesis and editing workloads.