SiamGPT: Quality-First Fine-Tuning for Stable Thai Text Generation
Abstract: Open-weights LLMs remain difficult to deploy for Thai due to unstable generation under complex instructions, despite strong English performance. To mitigate these limitations, We present SiamGPT-32B, an open-weights model based on Qwen3-32B, fine-tuned with a Quality-First strategy emphasizing curated supervision over data scale. The fine-tuning pipeline combines translated high-complexity English instruction data with a Thai-adapted AutoIF framework for instruction and linguistic constraints. Using supervised fine-tuning only, without continual pretraining or corpus expansion, SiamGPT-32B improves instruction adherence, multi-turn robustness, and linguistic stability. Evaluations on the SEA-HELM benchmark show that SiamGPT-32B achieves the strongest overall performance among similar-scale open-weights Thai models, with consistent gains in instruction following, multi-turn dialogue, and natural language understanding.
Summary
- The paper introduces SiamGPT-32B, a quality-first fine-tuned Thai LLM that significantly improves stability in multi-turn dialogues and instruction adherence.
- It employs a dual-stream data curation pipeline combining translation-based instruction transfer and a Thai-adapted AutoIF framework to curate a high-fidelity dataset.
- Empirical evaluations on SEA-HELM benchmarks demonstrate notable gains in code-switching stability and overall performance, validating the efficacy of controlled fine-tuning over large-scale noisy data.
SiamGPT-32B: Quality-First Fine-Tuning for Robust Thai Text Generation
Overview
The "SiamGPT: Quality-First Fine-Tuning for Stable Thai Text Generation" (2512.19455) paper introduces SiamGPT-32B, an open-weights Thai LLM derived from Qwen3-32B and designed to address reliability and linguistic stability in Thai text generation, particularly under complex instructions and multi-turn settings. The authors prioritize constraint-aware data curation over data scale, combining high-fidelity translation from English instruction corpora using gemma-3-27b-it with a Thai-adapted AutoIF deterministic verification pipeline. SiamGPT-32B is fine-tuned exclusively via supervised learning on a minimal, high-quality dataset, explicitly omitting continual pretraining or preference modeling. Empirical evaluations show consistent gains over leading Thai LLMs in stability, instruction adherence, and multi-turn dialogue competency.
Motivation and Data Curation Pipeline
The deployment of open-source LLMs in Thai production environments has been hampered by generation-time pathologies such as multilingual interference, code-switching, and instability under complex prompts. Existing Thai-centric adaptation efforts have improved benchmark metrics but fall short in robust generation for agentic settings where models act as final synthesizers.
SiamGPT-32B’s data curation pipeline is dual-stream:
- Stream 1: Translation-Based Instruction Transfer: Selected high-complexity English instruction datasets are translated into Thai using gemma-3-27b-it, leveraging its empirically best Thai translation scores on SEA-HELM. This preserves advanced reasoning and multi-turn scaffolding.
- Stream 2: Thai-Adapted AutoIF Constraint Enforcement: The AutoIF framework is localized with 39 custom Thai seed instructions targeting script-specific phenomena (vowel rules, consonant usage, lexical control), enabling deterministic, programmatic verification of instruction compliance.
After merging outputs from both streams, the final fine-tuning corpus comprises approximately 320,000 instruction-response pairs balanced for reasoning density and linguistic coverage.
Figure 1: SiamGPT data curation pipeline with dual-stream structure for high-fidelity Thai supervision.
Fine-Tuning Regimen
Fine-tuning is conducted via supervised next-token prediction on instruction–response pairs. Key design constraints include:
- Base Model: Qwen3-32B instruction-tuned checkpoint.
- Optimization: SFT only, excluding continual pretraining, RLHF, or DPO. Hyperparameters are tuned for stability via short-run, high-batch mixed precision training (64 × H100 GPUs, 7 hours, cosine schedule).
- Corpus: Only SystemChat-2.0 (translated) and AutoIF (English+Thai). No large-scale web crawl or noisy augmentation.
This regimen isolates the impact of curation and constraint, prioritizing controlled, error-resistant Thai output generation relevant for agentic and tool-augmented deployments.
Empirical Results
SiamGPT-32B is evaluated on the SEA-HELM Thai leaderboard and ancillary benchmarks. Notable findings include:
- Multi-Turn Dialogue Robustness: SEA-MTBench score increases from 57.94 (Qwen3-32B base) to 75.81, demonstrating significant contextual persistence improvement across conversational turns.
- Instruction Following: SEA-IFEval jumps from 75.47 to 83.00, reflecting consistent formatting and content compliance under task-specific constraints.
- Code-Switching Stability: Thai-only generation increases from 87.70 to 90.40, substantially reducing inadvertent multilingual artifacts.
- Natural Language Understanding: NLU (QA+sentiment) rises from 59.80 to 67.95, suggesting improved deeper comprehension from reasoning-dense supervision.
- Aggregate Performance: SiamGPT-32B sets the highest average SEA-HELM score within the open-weight 30B–32B Thai model class, outperforming Typhoon2.5-Instruct and OTG-R1, especially in controlled generation tasks, though NLG translation/summarization remains a relative weakness (intentional design trade-off).
Figure 2: SiamGPT-32B Main Results on SEA-HELM benchmarks, including code-switching stability, instruction following, and multi-turn dialogue.
Discussion and Implications
SiamGPT-32B advances the design of Thai LLMs for agentic deployments by demonstrating that minimalist, constraint-focused SFT outperforms brute-force data scaling in generation stability and instruction control. The model is particularly robust against code-switching and formatting/structural errors that often disrupt production use.
Notable limitations include factuality risk, decoding degeneration under creative prompts, cultural translation artifacts ("translationese"), and incomplete coverage of semantic drift or dialectal variation in stability metrics. The focus on controlled generation yields lower NLG fluency compared to models centered on naturalness and open-domain translation. Furthermore, resource requirements for reproduction are substantial, although the authors plan for lighter-weight variants.
The theoretical implication is a confirmation of the "Quality-First" paradigm: high-reasoning-density, constraint-aware supervision suffices for robust adaptation, and further scaling is counterproductive without additional controlling factors. Practically, these findings support specializing LLMs for high-value verticals (e.g., Thai tourism, regulatory domains) using compact, controllable datasets rather than broad, noisy corpora. Agentic workflow integration is facilitated by deterministic output behavior, especially in multi-step systems requiring exact Thai responses.
Prospective Research Directions
Key extensions proposed:
- Internalizing Thai cultural and domain-specific knowledge to minimize dependency on retrieval-augmented systems.
- Addressing NLG trade-offs by injecting more native Thai supervision targeting conversational and writing style.
- Formalizing corpus ablations to quantify contributions of each data stream and objective.
- Expanding evaluation protocols to stress tests, real-world user studies, and security/adversarial scenarios (prompt injection, misuse risks).
- Enhancing accessibility via open-source, parameter-efficient fine-tuning recipes.
Conclusion
SiamGPT-32B demonstrates that Thai LLMs optimized through quality-controlled supervised fine-tuning substantially outperform data-scale-heavy approaches on stability, instruction adherence, and agentic dialogue robustness. The model’s design principles—dual-stream high-fidelity curation and restriction to SFT—are empirically validated across diverse SEA-HELM benchmarks. For future AI development, this work advocates for domain-aligned, constraint-enforced fine-tuning strategies to build dependable, controllable LLMs for non-English deployment scenarios.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
Overview
This paper introduces SiamGPT-32B, a LLM fine-tuned to write Thai clearly and consistently. Its main goal is to reduce problems like mixing multiple languages in one answer (code-switching), not following instructions, and losing track in longer conversations. The model is open-weights, which means anyone can download and use it, and it’s built on top of another strong model called Qwen3-32B.
What questions did the researchers ask?
The researchers wanted to answer simple but important questions:
- How can we make an AI write Thai without accidentally mixing in English or other languages?
- How can we get the AI to follow rules and formats exactly (for example, “answer in bullet points” or “use only Thai words”)?
- How can we make the AI stay consistent and sensible over several turns in a chat (a longer back-and-forth conversation)?
- Can we reach these goals using a small, carefully chosen training set instead of a huge pile of mixed data?
How did they do it?
They used a “Quality-First” strategy. Think of training an AI like teaching a student: instead of giving the student a ton of messy notes, give them fewer but very well-written lessons and check their work carefully.
Here’s the approach, explained with everyday ideas:
- Carefully translated training: They took high-quality English instruction datasets (think: excellent study guides with step-by-step reasoning) and translated them into Thai using a strong translator model (Gemma-3-27b-it). This preserves complex thinking and multi-turn conversation patterns, but now in Thai.
- Rule-checking with AutoIF: They used a system called AutoIF like a strict teacher with a checklist. The AI’s answers are checked by small programs to make sure they follow rules exactly (formatting, language use, and Thai writing rules). They added Thai-specific checks (like vowel placement and valid Thai characters), so the AI learns to produce clean, correct Thai.
- Simple training recipe: They fine-tuned the model using supervised fine-tuning (SFT). In simple terms, the model reads examples of instructions and correct answers, then practices predicting the next word. They deliberately avoided more complicated methods (like extra pretraining or reward-based training) to keep the model stable and focused on following rules.
What did they find?
The model was tested with SEA-HELM, a benchmark designed for Southeast Asian languages. It includes tests for instruction following, multi-turn dialogue, understanding, translation, and safety.
Here are the key improvements, compared to the original Qwen3-32B:
- Multi-turn dialogue: Got much better at longer conversations (from about 58 to 76 on Thai MT-Bench). This means it stays on topic and keeps track of context more reliably.
- Instruction following: Improved at obeying rules (from about 75 to 83 on SEA-IFEval). Useful for tasks that require strict formats, like forms or step-by-step outputs.
- Thai-only stability (code-switching): Reduced language mixing (from about 88% stable to 90% stable). Answers are more purely Thai.
- Understanding (NLU): Better at comprehension tasks like questions and sentiment (from about 60 to 68).
- Thai exams: Slight improvement on Thai knowledge tests (from about 61 to 63).
- Overall score: Went up clearly (from about 68 to 77 on average).
They also compared SiamGPT-32B to other open Thai models of similar size (Typhoon2.5, OTG-R1). SiamGPT-32B had the best overall average score, especially strong in instruction following, reasoning, and safety. One trade-off: it wasn’t the top model for open-ended natural language generation (like translation fluency), likely because it focused on stability and rule-following rather than free-form style.
Why is this important?
- Better Thai chatbots and tools: More reliable answers in Thai without random English or other languages mixed in.
- Useful for real-world systems: In apps where the AI must follow strict formats or summarize tool outputs cleanly, this stability matters a lot.
- Open access: Because it’s open-weights, schools, startups, and researchers in Thailand can use and build on it without paying for a closed service.
- Shows a smart training strategy: You don’t always need more data—you need better data and good checks. This can help other language communities too.
Implications and next steps
- Impact: SiamGPT-32B makes Thai AI responses more dependable for customer service, education, government services, and tools that require correct formatting and clean language.
- Limitations: Like other AIs, it can still make mistakes or “hallucinate.” Because it learned from translated materials, some writing may sound less “naturally Thai” in style. It also isn’t the strongest at free-form generative tasks.
- Future plans: The team wants to add more native Thai conversational data, improve natural writing quality, and include more Thai cultural knowledge and domain topics (like tourism or local services) so the model can answer everyday Thai questions more accurately while keeping its strong stability and instruction-following skills.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
Below is a single, focused list of concrete gaps and unresolved questions that future researchers can act on:
- Publish detailed ablation tables quantifying how corpus size, data composition (translated SystemChat vs AutoIF English/Thai), and objectives (SFT vs DPO) trade off stability, instruction adherence, NLG, NLU, and multi-turn metrics; include multiple random seeds for robustness.
- Measure translation fidelity of the LLM-based Thai translation pipeline at conversation and reasoning-trace levels; quantify “translationese” artifacts; compare Gemma-3-27b-it against alternative translators and human post-editing; provide an error taxonomy with Thai-native annotations.
- Expand Thai-native supervision beyond the 39 seed instructions to systematically cover orthography, morphology, pragmatics (politeness particles, register), and regional/dialect variants; assess impact on naturalness and code-switching stability.
- Validate and release the Thai AutoIF verification scripts; quantify false positive/negative rates, coverage gaps, and whether constraint enforcement biases style or reduces generative diversity/creativity.
- Evaluate cross-lingual retention after Thai-focused fine-tuning: quantify performance changes in English and other languages, and assess stability in bilingual/mixed-language dialogues.
- Stress-test long-context and long-dialogue robustness beyond SEA-MTBench: measure consistency over 20–100 turns and 16k–32k tokens, tracking memory retention, persona persistence, and semantic drift.
- Define and operationalize “acceptable code-switching” (proper nouns, romanization, numerals, technical identifiers); build a Thai-specific mixed-script benchmark and update the stability metric accordingly.
- Quantify strict structured-output compliance (JSON/XML/CSV) under Thai prompts with programmatic validators; evaluate schema adherence and recoverability in tool-using agent pipelines.
- Conduct Thai-grounded RAG evaluations: measure evidence citation accuracy, source attribution, and hallucination rates under missing-context and adversarial probes; report failure modes.
- Perform Thai-focused safety and misuse evaluations: automated and human red-teaming for prompt injection, jailbreaks, and harmful content; report refusal consistency, false refusals, and bypass rates.
- Assess data contamination by running near-duplicate and semantic overlap checks between training sets and SEA-HELM/ThaiExam items; publish contamination statistics and decontaminated corpus variants.
- Analyze tokenization effects of Qwen3-32B on Thai (grapheme segmentation, vowel/consonant markers, tone diacritics); evaluate whether Thai-aware tokenization reduces code-switching and improves NLG.
- Diagnose the NLG gap: disentangle causes (translationese, lack of native Thai writing data, tokenizer issues); run controlled additions of Thai-native writing corpora and test multi-objective training with stability as a hard constraint.
- Systematically study Thai preference optimization (DPO/RLHF): identify reward models and objectives that boost naturalness without degrading stability or multi-turn control; publish Pareto frontiers.
- Evaluate domain generalization in Thai verticals (legal, medical, finance, tourism, navigation) with domain-specific metrics; quantify gains from domain datasets versus general supervision.
- Add blinded human evaluation with Thai annotators for naturalness, politeness, cultural appropriateness, and instruction fidelity; report inter-annotator agreement and correlation to LLM-as-a-judge scores.
- Resolve reproducibility inconsistencies (e.g., claimed warmup omission vs table showing 10% warmup); release training scripts, seeds, preprocessing/packing details, and VeRL configs for exact replication.
- Benchmark decoding pathologies under Thai prompts (repetition, looping) across decoding settings; publish recommended decoding defaults; evaluate unlikelihood training or entropy regularization as mitigations.
- Test multi-turn tool-use scenarios: assess robustness when synthesizing noisy tool outputs, handling schema changes, and managing errors; measure graceful degradation and recovery behaviors.
- Broaden evaluation coverage: include long-context stress suites, mixed-domain prompts, and Thai culture-aware instruction-following benchmarks (e.g., WangchanThaiInstruct); categorize failure types.
- Measure efficiency and deployment trade-offs: inference latency, memory footprint, throughput vs Qwen3-32B; explore parameter-efficient fine-tuning to reproduce gains in smaller models or lower compute budgets.
- Audit bias and fairness: identify Thai socio-cultural stereotypes and demographic skews in sentiment/toxicity tasks; quantify disparate impact and evaluate mitigation strategies.
- Clarify licensing and provenance for translated datasets and AutoIF outputs; document third-party content restrictions and implement provenance tracking pipelines.
- Quantify the impact of retaining English AutoIF seeds: does their inclusion prevent catastrophic forgetting, and how does it affect Thai style, stability, and cross-lingual behavior? Provide ablations removing vs retaining them.
- Verify conversation structure preservation in translation: ensure turn roles, system prompts, discourse markers, pronouns, and honorifics are correctly mapped to Thai; measure loss of context-dependent references.
Practical Applications
Overview
Based on SiamGPT-32B’s Quality-First fine-tuning, Thai-adapted AutoIF constraints, and demonstrated gains in instruction following, multi-turn robustness, and Thai-only generation stability, the following applications translate the paper’s findings into practical deployments and roadmaps. Each item notes key sectors, potential tools/products/workflows, and assumptions/dependencies that affect feasibility.
Immediate Applications
- Thai final-response generator for agentic pipelines
- Description: Use SiamGPT-32B as the “last hop” to synthesize tool outputs (retrieval, calculators, databases, APIs) into stable, fluent, Thai-only answers that respect strict formatting (JSON, bulleting, templates).
- Sectors: software, IT services, finance, healthcare, government digital services.
- Tools/Products/Workflows: RAG backends; orchestration frameworks (e.g., LangChain, LlamaIndex); JSON/XML schema validators; AutoIF-based output checkers; prompt templates; inference service on-prem or cloud (Hugging Face weights).
- Assumptions/Dependencies: High-quality upstream grounding to reduce hallucinations; guardrails to permit legitimate English tokens (URLs, proper nouns); prompt-injection defenses and output validation.
- Thai customer support chatbots and voicebots
- Description: Multi-turn, instruction-stable assistants that reduce code-switching in Thai for contact centers (chat and IVR), with consistent escalation, identity verification flows, and post-call summaries.
- Sectors: e-commerce, telco, banking, airlines, utilities, public-sector hotlines.
- Tools/Products/Workflows: ASR/TTS for Thai; conversation memory; ticketing integration (CRM); safety filters for refusal/handoff; decoding controls (temperature, repetition penalties).
- Assumptions/Dependencies: Human-in-the-loop for sensitive intents; call logging compliance; latency budgets for real-time speech.
- Structured document and form generation in Thai
- Description: High-precision generation of regulated forms, letters, and templates (e.g., bank statements, HR letters, government e-forms) with strict slot-filling and layout constraints.
- Sectors: government, finance, insurance, HR, legal.
- Tools/Products/Workflows: Template engines; schema conformance checks; AutoIF rule packs for formatting; e-signature workflows; approval queues.
- Assumptions/Dependencies: Human review for legal/compliance documents; clear exception handling for mixed-script fields (codes, IDs, names).
- Thai summarization and post-processing of tool outputs
- Description: Reliable Thai summaries of meetings, tickets, research snippets, and retrieval results; strong fit given observed gains in NLU and instruction adherence (not optimized for open-ended creative writing).
- Sectors: enterprise knowledge management, media, education.
- Tools/Products/Workflows: Meeting transcription; chunked summarization pipelines; “grounded answering” patterns that cite sources; redaction.
- Assumptions/Dependencies: Quality transcripts or retrieval context; source-citation prompts to curb hallucinations.
- Thai content moderation and safety tagging
- Description: Use improved Safety competency (SEA-HELM) for toxicity detection, policy classification, and refusal scaffolding in Thai.
- Sectors: social media, messaging, marketplaces, gaming, community platforms.
- Tools/Products/Workflows: Moderation queues; rule-based thresholds; human escalation; refusal-policy prompt libraries; automated red-teaming in Thai.
- Assumptions/Dependencies: Continuous evaluation against emerging adversarial prompts; oversight to calibrate false positives/negatives.
- Localization QA and style-control for Thai
- Description: Even if not SOTA on NLG/translation, SiamGPT-32B can act as a QA controller: detect code-mixing, enforce style guides, check controlled vocab/glossaries, and validate format preservation in Thai translations.
- Sectors: software localization, e-commerce catalogs, public-sector communications.
- Tools/Products/Workflows: Style-guide prompts; glossary checkers; code-switching metric as CI gate; AutoIF constraints for “do/don’t” lexical choices.
- Assumptions/Dependencies: Upstream MT engine for draft translations; curated glossaries and style rules; allowance for proper nouns/technical tokens.
- Open-weights, compliance-friendly Thai assistant (on-prem)
- Description: Deployable where data residency and vendor lock-in are concerns, with predictable behavior in Thai.
- Sectors: finance, healthcare, public sector, critical infrastructure.
- Tools/Products/Workflows: Self-hosted inference; audit logging; differential privacy at the application layer; RBAC around tools/APIs.
- Assumptions/Dependencies: MLOps capacity; GPU availability for target latency; internal security reviews.
- Anti–code-switching post-processor for Thai
- Description: Wrap other (multilingual) models with SiamGPT-32B to rewrite outputs into Thai-only text that preserves entities and structure.
- Sectors: cross-industry; media and CX teams.
- Tools/Products/Workflows: Rewriting microservice; AutoIF verification to enforce Thai script; exception lists for named entities/URLs.
- Assumptions/Dependencies: Added latency from two-stage generation; clear exception handling to prevent over-correction.
- Thai tutoring and exam-practice assistants (with grounding)
- Description: Multi-turn, instruction-following tutoring for Thai curricula, explanations, and practice questions; works best when answers are grounded to materials.
- Sectors: education (K-12, test prep, vocational).
- Tools/Products/Workflows: Curriculum-aligned RAG; step-by-step reasoning prompts; solution checkers; content filters.
- Assumptions/Dependencies: Modest gains on ThaiExam imply retrieval and vetted content are needed; disclaimers for high-stakes guidance.
- Research baselines and datasets for Thai instruction following
- Description: Immediate reuse of released datasets and recipe (translated SystemChat-2.0 Thai; AutoIF English+Thai) to study instruction adherence and stability in Thai and similar scripts.
- Sectors: academia, applied research, model evaluation vendors.
- Tools/Products/Workflows: SEA-HELM pipelines; code-switching scoring; ablation-ready SFT recipe; reproducible prompts.
- Assumptions/Dependencies: Access to base checkpoints and compute; dataset licenses and decontamination checks.
Long-Term Applications
- Thai “Native Expert” assistant with internalized cultural and geographic knowledge
- Description: Reduce dependence on retrieval for everyday Thai-local queries; improve cultural nuance and naturalness beyond translationese artifacts.
- Sectors: daily life, tourism, media, public services.
- Tools/Products/Workflows: Curated Thai-native corpora; multi-objective training with stability as a hard constraint; human evaluations with Thai speakers.
- Assumptions/Dependencies: High-quality native Thai datasets; IP/licensing; careful balance between NLG naturalness and stability.
- Domain-specialized Thai vertical models via parameter-efficient fine-tuning
- Description: PEFT/LoRA adapters for healthcare triage, legal drafting, banking ops, and tourism assistants.
- Sectors: healthcare, legal, finance, travel.
- Tools/Products/Workflows: Domain corpora; structured output schemas; safety red-teaming per domain; approval workflows.
- Assumptions/Dependencies: Domain data access; regulatory sign-off; clinical/legal oversight for high-stakes use.
- Closing the Thai NLG gap without sacrificing stability
- Description: Multi-objective or constrained optimization that improves translation/creative writing while maintaining Thai-only stability and instruction adherence.
- Sectors: localization, publishing, education.
- Tools/Products/Workflows: Thai-native conversational/writing supervision; naturalness metrics; style controllability; constrained decoding.
- Assumptions/Dependencies: New training/eval datasets; compute for objective tuning; risk of regressions managed via regression tests.
- Distilled or small Thai models for edge and on-device use
- Description: Compress SiamGPT-32B behavior into smaller footprints for kiosks, IVR gateways, rural clinics, and offline government counters.
- Sectors: customer service, public-sector field ops, retail.
- Tools/Products/Workflows: Knowledge distillation; quantization; latency profiling; fallback to cloud for complex queries.
- Assumptions/Dependencies: Acceptable quality/latency trade-offs; device constraints (memory, power); privacy requirements.
- Security- and safety-hardened Thai agent stack
- Description: Formalize defenses against prompt injection and data exfiltration; robust refusal and monitoring for Thai interactions.
- Sectors: finance, healthcare, government, enterprise IT.
- Tools/Products/Workflows: OWASP GenAI controls; HarmBench-style automated red-teaming; least-privilege tool access; policy sandboxes; audit trails.
- Assumptions/Dependencies: Security engineering capacity; continuous adversarial testing; incident response playbooks.
- Regional replication of the Quality-First pipeline to other Southeast Asian languages
- Description: Extend the dual-stream (translation + AutoIF constraints) approach to Lao, Khmer, Burmese, Vietnamese, etc.
- Sectors: multinational enterprises, NGOs, academia.
- Tools/Products/Workflows: Language-specific AutoIF rule packs; script-aware validators; regional SEA-HELM-style benchmarks.
- Assumptions/Dependencies: Native linguist involvement; script/orthography nuances; suitable translation backbones and evaluation sets.
- Guardrail SDK: AutoIF rule packs and code-switching gates for Thai
- Description: Productize constraint checkers as an SDK to validate formatting, style, and Thai-only outputs across models.
- Sectors: MLOps platforms, LLM app developers, QA vendors.
- Tools/Products/Workflows: Policy-as-code; CI/CD gates for model updates; telemetry on violation rates; exception catalogs.
- Assumptions/Dependencies: Developer adoption; maintenance of rules as policies evolve.
- Deployment-style evaluation suites for Thai
- Description: Long-context, multi-turn stress tests; “acceptable code-switching” regimes; real-world prompt distributions for Thai.
- Sectors: evaluation providers, research labs, enterprise QA.
- Tools/Products/Workflows: Scenario generators; human preference studies with Thai raters; regression dashboards across app releases.
- Assumptions/Dependencies: Annotation budgets; reproducible judging; dataset governance for contamination.
- Multimodal Thai assistants (speech/vision)
- Description: Combine stable Thai text with ASR/TTS and vision for kiosks, tourism guides, accessibility assistants.
- Sectors: tourism, transportation, healthcare accessibility, retail.
- Tools/Products/Workflows: Thai ASR/TTS; OCR for Thai scripts; multimodal LLMs; device integration.
- Assumptions/Dependencies: Multimodal datasets with Thai annotations; latency constraints; accessibility standards compliance.
- Policy and governance frameworks for open Thai LLMs
- Description: Guidance for public procurement, data residency, transparency, evaluation requirements, and language preservation goals.
- Sectors: government, regulators, public institutions.
- Tools/Products/Workflows: Standardized audits (safety, bias, robustness); procurement checklists; public model cards in Thai; citizen feedback loops.
- Assumptions/Dependencies: Inter-agency coordination; legal frameworks; funding for independent evaluation.
Notes on feasibility across applications:
- Strengths to leverage: instruction adherence, multi-turn robustness, Thai-only stability, improved NLU, open weights (on-prem options).
- Limitations to plan for: factuality/hallucinations (mitigate with RAG and citations), decoding pathologies under unconstrained creative tasks (use decoding controls), possible translationese style for native writing (add Thai-native data or QA passes), and compute requirements for fine-tuning or low-latency inference.
Glossary
- Agentic: Systems where models proactively plan and use tools to accomplish tasks. "agentic or tool-augmented systems"
- AutoIF: A programmatic framework that verifies instruction-following via executable constraints. "Thai-adapted AutoIF framework"
- BF16 mixed precision: Training arithmetic using bfloat16 to improve memory efficiency while maintaining numerical stability. "Training is performed using BF16 mixed precision for numerical stability and memory efficiency."
- Catastrophic forgetting: Loss of previously learned capabilities when fine-tuning on new data. "mitigate catastrophic forgetting"
- Code-switching: Mixing multiple languages or scripts within a single output. "we observed frequent code-switching"
- Code-Switching Score: A metric for Thai-only generation stability under Thai prompts. "As the Code-Switching Score increases from 87.70 to 90.40"
- Continual pretraining: Ongoing pretraining on large unlabeled corpora after initial training. "without continual pretraining"
- Deterministic verification: Rule-based checks that validate outputs against explicit constraints using code. "through deterministic verification"
- Direct Preference Optimization (DPO): A preference-based training method that optimizes models using paired comparisons. "Direct Preference Optimization (DPO)"
- Flash Attention: An efficient attention kernel that accelerates transformer attention and reduces memory use. "integrates Fully Sharded Data Parallelism (FSDP) with Flash Attention."
- Fully Sharded Data Parallelism (FSDP): A distributed training strategy that shards model states across GPUs. "Fully Sharded Data Parallelism (FSDP)"
- Gradient accumulation: Simulating a larger batch size by accumulating gradients over micro-batches before an update. "corresponds to a gradient accumulation factor of 4."
- Learning-rate warmup: Gradually increasing the learning rate at the start of training to stabilize optimization. "we omit learning-rate warmup"
- LLM-as-a-Judge: An evaluation paradigm where an LLM scores or judges model outputs. "following the LLM-as-a-Judge paradigm"
- Pareto-optimal: A best-effort trade-off where improving one objective would worsen another. "a minimalist, Pareto-optimal corpus"
- Quality-First strategy: A data curation philosophy prioritizing carefully curated, high-fidelity supervision over data scale. "Quality-First strategy"
- Retrieval-augmented generation: Enhancing outputs by grounding them in externally retrieved evidence. "Retrieval-augmented generation can reduce unsupported claims by grounding responses in external evidence"
- Reward modeling: Learning a scoring function over outputs to guide RL or preference optimization. "instability introduced by reward modeling"
- SEA-HELM: A benchmark suite for Southeast Asian languages covering NLU, NLG, NLR, instruction following, safety, and dialogue. "SEA-HELM benchmark suite"
- Sequence packing: Concatenating multiple shorter sequences to better utilize the context window. "Sequence packing is enabled to maximize token utilization."
- Supervised fine-tuning (SFT): Optimizing next-token prediction on labeled instruction–response data. "supervised fine-tuning (SFT)"
- Volcano Engine Reinforcement Learning (VeRL): A high-performance post-training framework for LLMs with flexible dataflows. "Volcano Engine Reinforcement Learning (VeRL)"
Collections
Sign up for free to add this paper to one or more collections.
