SiamGPT-32B: Fine-Tuned Thai LLM
- The paper introduces a supervised fine-tuning strategy that enhances Thai text stability by reducing code-switching and formatting errors in multi-turn dialogues.
- It employs a dual-stream corpus—translated SystemChat-2.0 and AutoIF—to create 320K high-quality instruction pairs that enforce deterministic language constraints.
- Evaluated on SEA-HELM benchmarks, SiamGPT-32B demonstrates significant improvements in instruction adherence, natural language understanding, and overall Thai response quality.
SiamGPT-32B is a 32-billion-parameter, open-weights LLM built by fine-tuning Qwen3-32B to function as a stable, instruction-sensitive Thai text generator. Its stated objective is narrowly defined: to produce fluent Thai-only text under complex, multi-turn instructions while avoiding code-switching, injected non-Thai tokens, and formatting failures that commonly appear in open-source LLM deployments for Thai. The model is presented as a supervised fine-tuning-only system, without continual pretraining, corpus expansion, RLHF, or DPO, and is evaluated primarily through SEA-HELM as a reliability-oriented Thai generation model rather than a general free-form generative model (Pairatsuppawat et al., 22 Dec 2025).
1. Problem setting and design target
SiamGPT-32B is framed around a specific deployment problem in Thai NLP: many strong open-source LLMs, including Qwen, Gemma, and Mistral, perform well in English but exhibit unstable Thai generation under complex instructions. The reported failure modes include frequent code-switching, injected non-Thai tokens, and multi-turn inconsistency. In production-style settings, particularly when upstream tool outputs must be synthesized into a final Thai response, these instabilities reduce reliability and can force practitioners toward proprietary APIs (Pairatsuppawat et al., 22 Dec 2025).
The model’s sole training target is correspondingly restrictive. It is optimized to produce fluent, Thai-only text, remain stable under complex multi-turn instructions, avoid multilingual interference, and rigidly adhere to formatting or content constraints. This positioning distinguishes SiamGPT-32B from Thai-centric efforts that rely on continual pretraining or large multilingual corpus expansion. A plausible implication is that the model is intended less as a broad-ended creative generator than as a controllable final-response component in constrained pipelines.
2. Quality-First fine-tuning philosophy
The central methodological claim is a “Quality-First” fine-tuning strategy, described as being inspired by Li et al. (2024). Instead of scaling raw instruction-response data or relying on reward-based tuning, the model is trained on a compact, curated supervised fine-tuning corpus designed to encode dense reasoning patterns and strong constraint enforcement. The paper contrasts this with more common approaches such as continual pretraining on web text, DPO, and large-scale noisy instruction collection, which the authors report can degrade stability or coherence in deployment (Pairatsuppawat et al., 22 Dec 2025).
Four benefits are attributed to this strategy on Thai generation tasks measured by SEA-HELM. Instruction adherence improves through deterministic constraint supervision; multi-turn robustness rises through curriculum-like translation of structured dialogues; linguistic stability increases through reduction of code-switching artifacts; and overall Thai naturalness is maintained, with a reported trade-off against open-ended generative fluency. The formulation is explicitly quality-over-scale: a compact, high-fidelity SFT corpus is treated as preferable to raw data volume for stable Thai response generation.
This emphasis also clarifies what SiamGPT-32B is not. It is not presented as a model whose capabilities derive from expanded monolingual pretraining or preference optimization. Rather, the paper argues that careful supervision design can materially alter deployment behavior even when the base architecture and tokenizer remain unchanged.
3. Data construction and constraint supervision
The training corpus is assembled from two parallel streams that produce high-fidelity instruction-response pairs. The first is a translation-based instruction-transfer pipeline built from high-quality English instruction datasets, including QuixiAI/SystemChat-2.0. Gemma-3-27b-it is used as the translation model and is reported to have achieved the highest SEA-HELM Thai translation score, 93.03. Using this model, 141,000 multi-turn conversations, each containing 3 to 25 turns, are translated wholesale into Thai while preserving reasoning traces and dialogue structure (Pairatsuppawat et al., 22 Dec 2025).
The second stream adapts the AutoIF framework of Dong et al. (2024) to Thai. AutoIF is described as programmatically verifying model outputs against explicit instruction and orthographic constraints with executable scripts. SiamGPT-32B retains the original 36 English AutoIF prompts to preserve universal instruction-following patterns and adds 39 hand-crafted Thai seed instructions targeting vowel placement, consonant rules, lexical form control, and other Thai-specific orthographic phenomena. This synthesis stage yields approximately 180,000 instruction-response pairs that are validated through deterministic checks such as script-only output and format compliance.
The two streams are merged into approximately 320,000 total pairs, specified as 141K translated SystemChat plus 180K AutoIF. All pairs pass programmatic verification, and no manual annotation or additional noisy data is introduced. The paper is explicit about exclusions: no continual pretraining, no RLHF, no DPO, and no broad web crawling or corpus expansion are used. This corpus assembly procedure is central to the model’s identity, because nearly all downstream claims are tied to the proposition that higher-fidelity supervision can outperform larger but noisier pipelines for Thai instruction following.
4. Architecture and implementation details
SiamGPT-32B preserves the underlying Qwen3-32B architecture, including the 32B parameter count, transformer blocks, tokenizer, and 8,192-token context window. The modification is confined to supervised fine-tuning on top of the instruction-tuned checkpoint. Training uses a standard next-token log-likelihood objective, described as cross-entropy over the target sequence, with BF16 mixed precision and sequence packing for GPU utilization (Pairatsuppawat et al., 22 Dec 2025).
The reported optimization configuration is unusually specific. Fine-tuning runs for 4,096 steps with a global batch size of 512 tokens, defined as 2 tokens per GPU × 64 GPUs with gradient accumulation × 4. The optimizer is AdamW with weight decay 0.01, gradient clipping at 1.0, and a cosine learning-rate schedule peaking at . No learning-rate warmup is used, in accordance with Qwen2.5’s post-training protocol. The run processes approximately 2.1 billion tokens in about seven hours on a cluster of 8 nodes × 8 NVIDIA H100 GPUs, each with 80 GB memory, using full FSDP2 sharding and Flash Attention v2 (Liger).
Inference and evaluation use standard greedy or sampling decoders. Code-switching is measured under forced Thai prompts. The paper does not provide explicit mathematical formulas for the loss, data-scoring metrics, or constraint-enforcement functions, and instead states that selection and filtering rules are implemented as deterministic scripts rather than formal scoring functions.
5. Evaluation protocol and reported performance
Evaluation is conducted through the official SEA-HELM pipeline across six competencies: Natural Language Understanding, Natural Language Generation, Natural Language Reasoning, Instruction Following, Safety, and Multi-Turn Dialogue. The task decomposition is specified as follows: NLU uses XQuAD extractive QA and Wisesight sentiment analysis; NLG uses bidirectional translation and abstract summarization via XLSum; NLR uses XNLI and causal reasoning tasks; Instruction Following uses SEA-IFEval, a Thai-localized IF-Eval; Safety uses SEA-HELM’s toxicity detection suite; and Multi-Turn Dialogue uses SEA-MTBench with GPT-4o as judge. An additional Code-Switching Score measures the fraction of Thai-prompted outputs containing only Thai script, excluding proper nouns, URLs, and technical tokens (Pairatsuppawat et al., 22 Dec 2025).
The baselines are the original Qwen3-32B checkpoint and two peer open-weights Thai models of similar scale: Typhoon2.5-Instruct (30B) and OTG-R1, described as DeepSeek-R1 32B. Against Qwen3-32B, SiamGPT-32B is reported to improve across all listed benchmarks.
| Metric | Qwen3-32B | SiamGPT-32B |
|---|---|---|
| Code-Switching (Stability) | 87.70 | 90.40 |
| SEA-IFEval | 75.47 | 83.00 |
| SEA-MTBench | 57.94 | 75.81 |
| ThaiExam | 61.40 | 63.00 |
| NLU (QA + Sentiment) | 59.80 | 67.95 |
| Average (across SEA-HELM) | 68.46 | 76.83 |
The paper reports the corresponding absolute gains as +2.70 for Code-Switching, +7.53 for SEA-IFEval, +17.87 for SEA-MTBench, +1.60 for ThaiExam, +8.15 for NLU, and +8.37 for the overall average. In the broader SEA-HELM leaderboard among 30–32B open Thai models, SiamGPT-32B ranks first overall with a 63.59 average, ahead of Typhoon2.5-Instruct at 60.44 and OTG-R1 at 55.28. Its strongest margins are reported in Instruction Following, Natural Language Reasoning, and Safety, while it concedes modestly in NLG, which the authors attribute to an emphasis on stability and constraint control over free-form fluency. All reported scores are absolute percentages, and no statistical tests are provided.
6. Ablations, limitations, and deployment implications
The paper states that detailed ablation tables are deferred to future work, but several summary findings are reported. Scaling the SFT corpus with mixed generic data degraded Thai generation stability and contextual coherence. Incorporating DPO produced only marginal instruction-following gains and introduced regressions elsewhere. The final corpus composition—translated SystemChat-2.0 plus AutoIF—is therefore described as a Pareto-optimal trade-off between scale and quality. No explicit model-size scaling studies beyond 32B are included (Pairatsuppawat et al., 22 Dec 2025).
Several failure modes are acknowledged. Internal testing identified translationese artifacts, defined operationally as unnatural phrasing introduced by machine translation; looping or repetition under creative prompts; and hallucinated factual claims when outputs are ungrounded. The model also inherits standard security concerns in tool-using contexts, including prompt injection. These limitations are important because the system’s strongest results appear in instruction adherence and stability, not in factual grounding or unconstrained generation.
The intended deployment niche follows directly from these trade-offs. SiamGPT-32B is positioned primarily as a final response generator in agentic or tool-augmented pipelines where strict format compliance, multi-turn consistency, and script stability are more important than maximal open-ended fluency. This suggests a design philosophy centered on operational reliability in Thai rather than general-purpose generative breadth. The model is released openly on Hugging Face under a standard open-source license, and the authors indicate plans for lighter fine-tuned variants using parameter-efficient tuning to improve reproducibility.
In aggregate, SiamGPT-32B exemplifies a supervised post-training strategy in which curated bilingual instruction transfer and programmatic Thai-specific constraint enforcement are treated as sufficient to improve Thai instruction following, multi-turn robustness, and linguistic stability in a 32B open-weights model without continual pretraining or reinforcement-based alignment (Pairatsuppawat et al., 22 Dec 2025).