---
title: 'KO-REAson-35B: Korean Reasoning Model'
url: https://www.emergentmind.com/topics/ko-reason-35b
type: topic
---

# KO-REAson-35B: Korean Reasoning Model

Searching arXiv for the exact KO-REAson paper and closely related 35B reasoning work for citation support.
KO-REAson-35B is a 35B-parameter Korean reasoning model in the KO-REAson series, built by post-training the A.X-3.1 35B instruction-tuned base model from SK Telecom. It is presented as the strongest model in a broader effort to build a high-performance reasoning system for a mid-resource language without relying on proprietary data or reinforcement-learning-heavy pipelines. Its defining technical feature is **Language-Mixed Chain-of-Thought**, a supervision scheme in which reasoning is carried out mainly in English as an anchor while preserving key Korean content and producing the final answer in fluent Korean. Within the reported evaluation suite, KO-REAson-35B achieves the highest overall average score, \(64.0_{2.5}\), ranking first on 5 of 9 Korean benchmarks and second on the remainder [2510.04230].

## 1. Identity, scope, and research position

KO-REAson-35B is the flagship model of the KO-REAson series and corresponds to the released model name **KO-REAson-AX3\_1-35B-1002**. It is released under **Apache 2.0**, matching the license of its base model, A.X-3.1. The model is situated within a training study spanning **nine models** from **4B to 35B parameters** across **six families**: Gemma-3, Qwen2.5, Llama-3.1, KONI-Llama-3.1, Kanana-1.5, and A.X-3.1 [2510.04230].

The model’s stated purpose is to address a specific failure mode in multilingual reasoning research: most prior reasoning-distillation pipelines are English-centric, and translated corpora can introduce artifacts and weak robustness on naturally occurring Korean inputs. KO-REAson-35B therefore uses **native Korean prompts** and **language-mixed long CoT supervision** rather than a Korean-only reasoning format or a translation-heavy synthetic pipeline. This positioning is important because the model is not framed as a generic multilingual chatbot; it is a Korean-focused reasoning system whose training design attempts to preserve both Korean semantic fidelity and the stronger reasoning behavior associated with English-pretrained distributions.

A central nuance is that KO-REAson-35B is not described as an RL-trained reasoning model. The paper explicitly characterizes it as an **SFT-only** system. This distinguishes it from contemporary pipelines that depend on preference optimization, RLHF, GRPO, RLVR, or other reinforcement-learning-heavy post-training stages. A plausible implication is that the contribution lies less in optimizer novelty than in data construction, supervision design, and curation.

## 2. Language-Mixed Chain-of-Thought

The core methodological contribution is **Language-Mixed CoT**. Instead of requiring the reasoning trace to be entirely in Korean or entirely in English, the teacher is prompted to produce a mixed-language reasoning process in which **English** supplies most of the logical scaffolding while **Korean** is preserved for prompt-grounded content such as named entities, quoted spans, and key terms [2510.04230].

The system prompt used for trace generation is given exactly as:

> **Think carefully, do not translate the question while solving. Preserve the question in Korean so that you keep all details without adding noise. After you finish thinking, state your answer in fluent and coherent Korean.**

The rationale is empirical. The paper argues that **English-only CoT** introduces translation noise and semantic drift, because long reasoning chains can gradually lose fidelity to the original Korean phrasing. Conversely, **Korean-only CoT** weakens reasoning quality, apparently because many open base models and teachers are stronger for abstract reasoning in English-pretrained distributions. Language-Mixed CoT is intended to combine the strengths of both regimes: English-centered reasoning with Korean semantic anchoring.

This design is enforced operationally. After generation, traces are discarded if their **Korean-character ratio is outside 5% to 20%**, ensuring that the traces are actually mixed rather than effectively monolingual. The paper does **not** provide a formal mathematical definition, objective function, or pseudocode specifically for Language-Mixed CoT.

Direct comparisons against monolingual supervision styles are reported. For **Gemma-3-4B**, the three-way comparison is:

- English: HRB 50.3, MCLM 48.1, KMMLU-Redux 52.2  
- Korean: HRB 40.6, MCLM 25.6, KMMLU-Redux 42.5  
- Language-Mixed: HRB 54.9, MCLM 55.8, KMMLU-Redux 53.0

For **Kanana-1.5-8B**, the comparison is:

- English: HRB 66.2, MCLM 60.5, KMMLU-Redux 64.0  
- Korean: 67.2, 31.8, 53.4  
- Language-Mixed: 74.6, 57.4, 64.4

These results support three conclusions emphasized in the paper: Korean-only CoT tends to damage reasoning-heavy performance, English-only CoT preserves reasoning better but is weaker on Korean-specific content, and Language-Mixed CoT is generally the best compromise. The authors further claim that this mixed format also contributes to **cross-lingual transfer** and **multimodal gains**, although the multimodal evidence is shown on another model rather than on KO-REAson-35B itself.

## 3. Data foundation: Yi-Sang and Yi-Sang-HQ

KO-REAson-35B is trained on **Yi-Sang-HQ**, a curated subset of the larger **Yi-Sang** Korean reasoning corpus. The full Yi-Sang resource consists of **5.79M native Korean prompts**, **3.7M long reasoning traces**, and **6.77B tokens** overall. The final KO-REAson models do not use the entire 3.7M-trace set; they use the smaller, high-yield **260k** Yi-Sang-HQ subset [2510.04230].

The data pipeline begins with web collection from public Korean Q&A and community sites. The authors compiled **54 candidate sites** and assigned each a legal category: **A** for crawling and redistribution permitted, **B** for crawling allowed but commercial use or redistribution prohibited, and **C** for crawling prohibited. They then excluded category C sites, low-volume sources, heavily obfuscated sites, and near-duplicate sources, removing **26 websites** at that stage. Data from category B sites was used for training and analysis but not redistributed.

Prompt filtering is deliberately light. Prompts are kept **verbatim**, without template normalization or rewriting, in order to preserve typos, abbreviations, mixed script, and internet style. Filtering removes prompts with **Korean-character ratio < 30%**, length **< 50 characters**, or length **> 8,192 characters**. Because the web-collected pool lacked enough competition-style problems, additional prompts were taken from **OpenThought** and translated into Korean using **Gemini-2.5-Flash**. The paper notes that earlier translation attempts using GPT-4o-mini, Qwen2.5-72B-Instruct, and Gemini-2.0-Flash caused training instability.

The reasoning traces are not based on scraped web answers. Instead, the authors **regenerate all targets from the prompt alone**. They considered and rejected agreement sampling with multiple samples plus an LLM judge, as well as hint-based refinement using scraped answers, citing compute cost, leakage risk, artifact risk, distribution shift, and answer noisiness. The main teacher for trace generation is **Qwen3-32B**, and ablations show that **Qwen3-32B with long Language-Mixed CoT** is better than Qwen3-4B and better than short-CoT or solution-only supervision.

Yi-Sang-HQ is shaped by targeted filtering. The paper removes:

1. degeneration cases with endless repetition,  
2. samples containing multiple `<think> ... </think>` blocks,  
3. samples whose final solution after `</think>` is in a non-Korean language,  
4. instances longer than **16k tokens**,  
5. benchmark-near-duplicates using an **\(n\)-gram similarity filter with \(n=13\)** against held-in and held-out benchmarks.

The final **260k** subset contains **62k OpenThought**, **86k Code**, **37k Science**, and **66k Exams**. The authors also apply two augmentations, mainly on exam-style data: **style augmentation**, which adds directives such as returning the final answer in `\boxed{}` format, and **option augmentation**, which uses **BM25** retrieval over the exam pool to find similar questions and merge distractor options, subject to negation filtering, near-duplicate removal, a cap of **10 options**, and preservation of the original correct option. The paper reports that both augmentations help, and neither dominates consistently.

## 4. Training recipe and implementation

KO-REAson-35B is initialized from **A.X-3.1-35B**, described as a LLaMA-style model family from SK Telecom, focused particularly on Korean, trained on approximately **2.1 trillion tokens**, and strong on Korean benchmarks while still performing well in English [2510.04230].

The model is trained by **supervised fine-tuning only**. The paper explicitly states several negative facts: there is **no continued pretraining**, **no preference optimization**, **no DPO**, **no RLHF**, **no GRPO**, **no RLVR**, and no explicit curriculum learning beyond the subset-selection process. The final supervision format is:

- Korean prompt  
- language-mixed long CoT trace  
- Korean final answer

All experiments use **5 epochs unless otherwise specified**, but **A.X-3.1-35B is trained for 3 epochs due to computational constraints**. KO-REAson-35B therefore receives **3 epochs** on Yi-Sang-HQ.

The reported training hyperparameters are:

- precision: **bfloat16**  
- optimizer: **AdamW**  
- learning rate: **\(2\times10^{-5}\)**  
- warmup: **10%**  
- weight decay: **\(1\times10^{-5}\)**  
- final global batch size: **512**

The implementation uses **PyTorch FSDP**, **Liger kernels**, and **FlashAttention-2**. A packing ablation on smaller models found that packing improved speed but somewhat worsened benchmark performance; as a result, **all reported models are trained without packing**, including KO-REAson-35B.

The exact training sequence length is **not specified**. The paper does state that Yi-Sang-HQ drops traces longer than **16k tokens**, and that evaluation uses `max_tokens=32,768`, but it does not give the training context limit for the 35B run. Hardware spans **8 NVIDIA H100** to **24 NVIDIA H200 GPUs**, but the exact allocation or total compute specific to KO-REAson-35B is not reported.

## 5. Evaluation protocol and reported performance

Evaluation is conducted with **vLLM** using **temperature \(=0.7\)**, **top\_p \(=0.9\)**, and **max\_tokens \(=32,768\)**. Models are instructed to place the final answer in `\boxed{...}`, and scoring extracts the **first valid boxed answer after the hidden think section**; if no valid boxed answer appears, the item is counted wrong. Main results are reported as **mean \(\pm\) standard error over \(n=3\) runs** [2510.04230].

The main Korean evaluation suite contains nine benchmarks: KMMLU-Redux, KMMLU-Pro, KMMLU-Hard, Math (Ko), AIME2024 (Ko), KSM, HAE-RAE Bench (HRB), CLIcK, and KoBALT-700. KO-REAson-35B’s reported scores are:

| Benchmark | Score | Rank |
|---|---:|---:|
| KMMLU-Redux | \(76.0_{0.4}\) | 1st |
| KMMLU-Pro | \(47.4_{0.6}\) | 2nd |
| KMMLU-Hard | \(51.4_{0.5}\) | 1st |
| Math (Ko) | \(87.5_{0.6}\) | 1st |
| AIME2024 (Ko) | \(66.7_{11.5}\) | 2nd |
| KSM | \(65.7_{8.6}\) | effectively 2nd/tied for 2nd |
| HRB | \(78.9_{0.7}\) | 1st |
| CLIcK | \(70.9_{0.3}\) | tied 1st |
| KoBALT-700 | \(34.9_{1.0}\) | 2nd |

Its overall average is **\(64.0_{2.5}\)**, the best among the compared systems. The strongest baseline average is **QwQ-32B** at **\(59.6_{3.1}\)**, so KO-REAson-35B exceeds it by **4.4 points**. Other reported baseline averages are **GPT-OSS-20B: \(58.8_{1.2}\)**, **DeepSeek-R1-32B: \(56.4_{4.5}\)**, and **EXAONE-Deep-32B: \(57.4_{5.2}\)**.

Comparison against its own backbone reveals the model’s specialization profile. Relative to **A.X-3.1-35B**, KO-REAson-35B improves substantially on reasoning-heavy tasks: **KMMLU-Hard** rises from \(44.2_{0.2}\) to \(51.4_{0.5}\), **Math** from \(73.1_{2.1}\) to \(84.5_{0.6}\), **AIME24** from \(16.7_{3.3}\) to \(66.7_{11.5}\), and **KSM** from \(26.8_{0.8}\) to \(65.7_{8.6}\). At the same time, the base model remains stronger on some Korean cultural or linguistic benchmarks: **HRB** falls from \(84.0_{0.5}\) to \(78.9_{0.7}\), **CLIcK** from \(76.6_{0.6}\) to \(70.9_{0.3}\), and **KoBALT** from \(35.5_{0.2}\) to \(34.9_{1.0}\). This makes KO-REAson-35B a reasoning-optimized specialization rather than a uniform improvement over the backbone.

The paper also reports **cross-lingual evaluation** on English benchmarks. Despite training on Korean prompts only, KO-REAson-35B improves over A.X-3.1-35B from **AIME24 \(15.6_{3.8}\) to \(58.9_{16.4}\)**, from **AIME25 \(15.6_{1.9}\) to \(53.3_{12.0}\)**, and from **GPQA \(37.0_{0.8}\) to \(47.8_{5.3}\)**. The authors attribute this to universal math and science knowledge transfer and to the retention of English within Language-Mixed CoT. By contrast, the multimodal experiments in the paper are performed on **Gemma-3-12B / KO-REAson-12B**, not on the 35B model.

## 6. Interpretive significance, caveats, and release

KO-REAson-35B is most significant as evidence that a Korean reasoning model can reach top-tier benchmark performance through **curated native-language data**, **long CoT distillation**, and **Language-Mixed CoT**, without RL-based post-training. The paper’s broader empirical claim is that similar gains hold across the full **4B–35B** model range, with an average improvement of **\(+18.6\) points across nine benchmarks** for smaller and mid-sized models [2510.04230].

Several common misconceptions are directly contradicted by the training setup. First, KO-REAson-35B is **not** a Korean-only reasoning model: its hidden trace distribution is intentionally mixed, with English functioning as the anchor language. Second, it is **not** an RL-trained model: the pipeline is explicitly **SFT-only**. Third, it is **not** the paper’s multimodal flagship: the multimodal analysis is demonstrated on KO-REAson-12B rather than on the 35B system.

The limitations are also explicit. On **AIME2024 (Ko)** and **KSM**, KO-REAson-35B remains second rather than first, and the paper attributes this partly to limited competition-style data, with only about **60k filtered translated OpenThought items** retained in the final mixture. The pipeline depends heavily on **Qwen3-32B** as the trace generator. The subset-construction loop repeatedly used held-in benchmarks, which the authors acknowledge introduces some contamination risk. Exact training sequence length, exact 35B hardware allocation, and total compute are not specified. The paper also provides **no direct human evaluation**, relying instead on benchmark-based evidence.

Release details are clear. The authors state that they release the **data-curation pipeline**, **evaluation system**, **datasets**, and **models** at **https://huggingface.co/KOREAson**. **KO-REAson-35B / KO-REAson-AX3\_1-35B-1002** is released under **Apache 2.0**, while **Yi-Sang** and **Yi-Sang-HQ** are released under the **MIT License**. Some crawled data originated from category B sites and was used for training and analysis but was **not redistributed**.

In aggregate, KO-REAson-35B is best understood as a reasoning-specialized Korean post-trained model whose main contribution is methodological rather than architectural: it shows that **native Korean prompts**, **teacher-generated long reasoning traces**, and **Language-Mixed CoT** can produce a strong open 35B reasoning model with both Korean benchmark leadership and nontrivial English transfer, while also exposing the tradeoff between reasoning gains and some aspects of culturally or linguistically specialized Korean performance.

Source: https://www.emergentmind.com/topics/ko-reason-35b