Thai-SUP: Thai Multitask Spoken Language Understanding

• Thai-SUP is a data-generation pipeline that creates synthetic, task-supervised Thai speech data for multitask spoken language understanding.
• It leverages an LLM-based transfer pipeline employing augmentation, translation, and TTS to convert high-resource text supervision into spoken Thai.
• Integrated with XLSR-Thai and U-Align, Thai-SUP addresses data scarcity by providing training data for intent, NER, and speech rephrasing tasks.

Searching arXiv for the primary Thai-SUP paper and closely related Thai-resource papers to ground the article. Thai-SUP is a pipeline for generating Thai spoken language understanding data and the resulting open Thai spoken language understanding dataset introduced in the context of low-resource speech LLMs (SLLMs). It was proposed alongside XLSR-Thai, a Thai self-supervised speech encoder, and U-Align, a speech–text alignment method, as part of a unified approach to Thai multitask speech understanding. Within that system, Thai-SUP addresses the scarcity of task-supervised Thai speech data by transferring supervision from high-resource text understanding corpora into Thai spoken form, yielding the first open Thai spoken language understanding dataset of over 1,000 hours (Shao et al., 18 Sep 2025).

1. Definition and scope

Thai-SUP is the data-generation component of a Thai SLLM pipeline designed for multitask spoken language understanding rather than for automatic speech recognition alone. Its purpose is to create paired Thai speech–text supervision for higher-level tasks such as intent classification, named entity recognition, and speech rephrasing. The paper situates Thai-SUP as the response to a specific bottleneck: Thai is low-resource not only in raw speech pretraining data, but especially in paired speech-text understanding data (Shao et al., 18 Sep 2025).

This positioning is important because Thai-SUP is not a general Thai NLP benchmark, a text-only corpus, or a conventional transcription dataset. The paper explicitly treats it as a resource for spoken understanding tasks, and states that it is not merely a transcription corpus because it includes task-specific supervision that allows the SLLM to learn higher-level understanding behaviors. This distinguishes Thai-SUP from other Thai resources with different objectives, such as TUD, which is a UD-annotated treebank for supervised dependency parsing (Sriwirote et al., 2024), and Mangosteen, which is a Thai-language pretraining corpus rather than a task-labeled supervised dataset (Phatthiyaphaibun et al., 19 Jul 2025).

The term therefore denotes both a methodology and a dataset. As methodology, it is an LLM-based transfer pipeline that converts high-resource supervision into Thai spoken examples. As dataset, it is the concrete multitask corpus used in the paper’s second-stage fine-tuning for Thai spoken language understanding (Shao et al., 18 Sep 2025).

2. Problem setting and motivation

The motivating claim is that Thai SLLMs are limited by three issues: weak encoders for Thai, expensive ASR-based alignment, and scarcity of supervised spoken understanding corpora. Thai-SUP addresses the third issue. In the paper’s architecture, XLSR-Thai improves Thai speech representations, U-Align aligns speech embeddings with LLM text embeddings, and Thai-SUP supplies the multitask supervision required for downstream spoken understanding (Shao et al., 18 Sep 2025).

The targeted tasks illustrate why the resource is needed. The paper identifies intent classification, named entity recognition, and speech rephrasing as representative spoken understanding problems for which Thai lacks scalable supervised corpora. Manual collection of Thai speech with task-specific labels is described as costly and difficult to obtain at scale. Thai-SUP is therefore designed around supervision transfer rather than manual annotation: it starts from existing high-resource data and converts that supervision into Thai spoken form (Shao et al., 18 Sep 2025).

This design also reflects a broader shift from ASR-centric speech modeling to multitask understanding. A plausible implication is that the resource is intended to move Thai speech systems from transcription competence toward task-conditioned semantic competence. The paper states this directly in functional terms: Thai-SUP is essential for moving from speech recognition to multitask understanding (Shao et al., 18 Sep 2025).

3. Data-generation methodology

Thai-SUP is constructed through a multi-step transfer pipeline. For intent classification and named entity recognition, the process begins with high-resource English text understanding datasets and converts them into Thai spoken-language supervision through augmentation, filtering, translation, colloquialization, and text-to-speech synthesis. For intent classification, the source dataset is SNIPS. For named entity recognition, the sources are WikiANN and CONLL-2023 (Shao et al., 18 Sep 2025).

Each original example is augmented by DeepSeek-v3, which generates ten synthetic variants per instance. These candidates are then filtered using Gemini-2.5-flash to remove examples deemed unsuitable for downstream speech tasks. The retained English examples are translated into colloquial, spoken-style Thai rather than literal written Thai, and the translated outputs are synthesized into audio using a Thai fine-tuned LLaSa TTS model. The result is a paired speech–text corpus suitable for end-to-end spoken understanding training (Shao et al., 18 Sep 2025).

Speech rephrasing follows a different path. Instead of starting from text-only corpora, Thai-SUP mines existing ASR speech-text pairs that are good candidates for paraphrasing. DeepSeek-v3 is used to select appropriate ASR pairs, and Gemini-2.5-flash generates rewritten labels for the rephrased outputs. The paper characterizes the overall system as LLM-based data augmentation plus translation followed by TTS, with task-specific handling for speech rephrasing (Shao et al., 18 Sep 2025).

Several design choices are presented as quality controls rather than mere scaling heuristics. The use of ten synthetic variants per seed example is intended to increase diversity. Filtering with Gemini-2.5-flash is intended to reduce poor-quality examples. Translation into colloquial spoken Thai is a deliberate step to improve naturalness and TTS compatibility. The Thai-fine-tuned LLaSa TTS system is intended to improve audio realism. The paper argues that these choices improve transferability to spoken inputs, although it also notes that the resulting data remain synthetic (Shao et al., 18 Sep 2025).

4. Task coverage and dataset composition

The reported output of Thai-SUP is a multitask spoken language understanding dataset covering three core tasks. The paper gives the following composition (Shao et al., 18 Sep 2025):

Task	Source basis	Reported scale
Intent classification (IC)	SNIPS-derived examples	175 hours
Named entity recognition (NER)	WikiANN and CONLL-2023-derived examples	648 hours
Speech rephrasing (SR)	Selected ASR pairs and rewritten labels	more than 250 hours

Together these components yield over 1,000 hours total. The paper describes this as the first open-source Thai spoken language understanding dataset over 1,000 hours (Shao et al., 18 Sep 2025).

The task inventory is narrow but explicit. IC provides utterance-level semantic intent supervision. NER provides sequence-level entity supervision in spoken form. SR adds a generative reformulation task built from mined ASR pairs rather than translated text-understanding corpora. This task mix gives the dataset both classification-style and generation-style supervision. A plausible implication is that Thai-SUP is intended to train a generalist spoken understanding system rather than a single-task classifier (Shao et al., 18 Sep 2025).

The composition also clarifies what Thai-SUP does not cover. The paper notes that it is built from a small set of source task families and from English-origin corpora. It is therefore strong for IC, NER, and SR, but not necessarily comprehensive for all spoken understanding domains (Shao et al., 18 Sep 2025).

5. Model integration: XLSR-Thai and U-Align

Thai-SUP is introduced as one component of a three-part system. XLSR-Thai is the encoder-side foundation; U-Align is the alignment mechanism; Thai-SUP is the task-side supervision. XLSR-Thai is obtained by continuing XLSR pretraining on 36,000 hours of Thai speech, consisting of 16,000 hours of public Thai data plus 20,000 hours of in-house unlabeled Thai speech. U-Align then bridges the speech encoder and the frozen Typhoon2-LLaMa2-3B LLM through a two-stage procedure: Stage 1 learns modality alignment from paired speech and transcription, and Stage 2 fine-tunes the frozen LLM with task prompts and speech embeddings on spoken-language understanding data, including Thai-SUP (Shao et al., 18 Sep 2025).

A key technical point is that U-Align does not use the conventional ASR-based alignment paradigm in which the entire SLLM is trained through ASR. Instead, it directly aligns adapted speech embeddings with frozen LLM text embeddings. Let $H=\{h_i\}_{i=1}^I$ be adapted speech embeddings and $E=\{e_j\}_{j=1}^J$ be frozen LLM text embeddings. The pairwise cost is

$$C_{ij}=1-\frac{\langle h_i,e_j\rangle}{\|h_i\|\,\|e_j\|}.$$

The DTW loss is

$$\mathcal{L}_{\text{DTW-loss}}=\frac{1}{|\pi^\star|}\min_{\pi\in\mathcal{P}} \sum_{(i,j)\in \pi} C_{ij},$$

where $\mathcal{P}$ is the set of monotonic warping paths and $\pi^\star$ is the optimal path. The paper notes that normalization by $|\pi^\star|$ avoids sequence-length bias. It also reports that CTC can be used as an alternative alignment loss, but DTW performs better (Shao et al., 18 Sep 2025).

Thai-SUP becomes most useful after this alignment stage. The paper’s framing is that synthetic speech pairs alone are insufficient unless the model has first learned to treat speech as text-like embeddings in the LLM space. This suggests that Thai-SUP is not a stand-alone resource in the paper’s empirical logic; it is the supervision layer in a pipeline whose effectiveness depends on both Thai acoustic representation quality and cross-modal alignment (Shao et al., 18 Sep 2025).

6. Empirical findings, limitations, and significance

The experimental setup first aligns on 2,000 hours of ASR data from GigaSpeech2, MSR-86K, and Common Voice, and then fine-tunes with Thai-SUP for multitask understanding. The main comparison includes Whisper + ASR-based alignment, Whisper + U-Align (DTW), XLSR-Thai + Directly-MT, XLSR-Thai + ASR-based alignment, XLSR-Thai + U-Align (CTC), and XLSR-Thai + U-Align (DTW). The best overall configuration is XLSR-Thai + U-Align (DTW), which achieves 89.68 ACC on IC, 53.77 on overall NER, 90.91 on SR, and 13.32 CER on ASR. The corresponding XLSR-Thai + ASR-based alignment result is 81.71 on IC, 43.23 on NER, 87.91 on SR, and 13.81 CER on ASR (Shao et al., 18 Sep 2025).

These results are presented as evidence that Thai-SUP is effective when combined with a stronger Thai encoder and a better alignment strategy. The paper’s central claim is not that Thai-SUP alone solves Thai spoken understanding, but that Thai-SUP becomes crucial inside a system where XLSR-Thai and U-Align already address encoder quality and cross-modal alignment. Compared with Whisper-based systems, XLSR-Thai consistently performs better, reinforcing the dependence of Thai-SUP’s usefulness on the surrounding architecture (Shao et al., 18 Sep 2025).

The paper also states several limitations. Because Thai-SUP is synthetic, it inherits biases and errors from augmentation, translation, filtering, and TTS. Its quality depends on the adequacy of the source tasks, the correctness of LLM-generated labels, the faithfulness of translation into colloquial Thai, and the naturalness of synthesized speech. A further limitation is its reliance on a small set of source task families and on English-origin corpora. Another design dependency is that U-Align uses a subset of 2,000 hours of ASR data for alignment, so Thai-SUP’s effectiveness presupposes sufficient paired speech-text data before multitask tuning (Shao et al., 18 Sep 2025).

Within Thai NLP and speech research, Thai-SUP is significant because it supplies a previously missing layer of supervision: open, task-labeled Thai speech data for spoken understanding rather than transcription. The paper argues that the overall recipe—SSL encoder adaptation, universal speech–text alignment, and synthetic spoken supervision—is language-agnostic and can be extended to other low-resource languages. This suggests that Thai-SUP is best understood not only as a Thai dataset release, but also as an instantiation of a broader supervision-transfer framework for low-resource SLLMs (Shao et al., 18 Sep 2025).