KO-REAson-35B: Korean Reasoning Model

• KO-REAson-35B is a 35B-parameter Korean reasoning model that employs Language-Mixed Chain-of-Thought to blend English logical reasoning with Korean semantic fidelity.
• It is trained exclusively via supervised fine-tuning on curated native Korean prompts and high-quality Yi-Sang-HQ data, achieving top scores on several benchmarks.
• The model showcases effective cross-lingual transfer and practical reasoning improvements while avoiding reinforcement-learning-based post-training methods.

Searching arXiv for the exact KO-REAson paper and closely related 35B reasoning work for citation support. KO-REAson-35B is a 35B-parameter Korean reasoning model in the KO-REAson series, built by post-training the A.X-3.1 35B instruction-tuned base model from SK Telecom. It is presented as the strongest model in a broader effort to build a high-performance reasoning system for a mid-resource language without relying on proprietary data or reinforcement-learning-heavy pipelines. Its defining technical feature is Language-Mixed Chain-of-Thought, a supervision scheme in which reasoning is carried out mainly in English as an anchor while preserving key Korean content and producing the final answer in fluent Korean. Within the reported evaluation suite, KO-REAson-35B achieves the highest overall average score, $64.0_{2.5}$, ranking first on 5 of 9 Korean benchmarks and second on the remainder (Son et al., 5 Oct 2025).

1. Identity, scope, and research position

KO-REAson-35B is the flagship model of the KO-REAson series and corresponds to the released model name KO-REAson-AX3_1-35B-1002. It is released under Apache 2.0, matching the license of its base model, A.X-3.1. The model is situated within a training study spanning nine models from 4B to 35B parameters across six families: Gemma-3, Qwen2.5, Llama-3.1, KONI-Llama-3.1, Kanana-1.5, and A.X-3.1 (Son et al., 5 Oct 2025).

The model’s stated purpose is to address a specific failure mode in multilingual reasoning research: most prior reasoning-distillation pipelines are English-centric, and translated corpora can introduce artifacts and weak robustness on naturally occurring Korean inputs. KO-REAson-35B therefore uses native Korean prompts and language-mixed long CoT supervision rather than a Korean-only reasoning format or a translation-heavy synthetic pipeline. This positioning is important because the model is not framed as a generic multilingual chatbot; it is a Korean-focused reasoning system whose training design attempts to preserve both Korean semantic fidelity and the stronger reasoning behavior associated with English-pretrained distributions.

A central nuance is that KO-REAson-35B is not described as an RL-trained reasoning model. The paper explicitly characterizes it as an SFT-only system. This distinguishes it from contemporary pipelines that depend on preference optimization, RLHF, GRPO, RLVR, or other reinforcement-learning-heavy post-training stages. A plausible implication is that the contribution lies less in optimizer novelty than in data construction, supervision design, and curation.

2. Language-Mixed Chain-of-Thought

The core methodological contribution is Language-Mixed CoT. Instead of requiring the reasoning trace to be entirely in Korean or entirely in English, the teacher is prompted to produce a mixed-language reasoning process in which English supplies most of the logical scaffolding while Korean is preserved for prompt-grounded content such as named entities, quoted spans, and key terms (Son et al., 5 Oct 2025).

The system prompt used for trace generation is given exactly as:

Think carefully, do not translate the question while solving. Preserve the question in Korean so that you keep all details without adding noise. After you finish thinking, state your answer in fluent and coherent Korean.

The rationale is empirical. The paper argues that English-only CoT introduces translation noise and semantic drift, because long reasoning chains can gradually lose fidelity to the original Korean phrasing. Conversely, Korean-only CoT weakens reasoning quality, apparently because many open base models and teachers are stronger for abstract reasoning in English-pretrained distributions. Language-Mixed CoT is intended to combine the strengths of both regimes: English-centered reasoning with Korean semantic anchoring.

This design is enforced operationally. After generation, traces are discarded if their Korean-character ratio is outside 5% to 20%, ensuring that the traces are actually mixed rather than effectively monolingual. The paper does not provide a formal mathematical definition, objective function, or pseudocode specifically for Language-Mixed CoT.

Direct comparisons against monolingual supervision styles are reported. For Gemma-3-4B, the three-way comparison is:

• English: HRB 50.3, MCLM 48.1, KMMLU-Redux 52.2
• Korean: HRB 40.6, MCLM 25.6, KMMLU-Redux 42.5
• Language-Mixed: HRB 54.9, MCLM 55.8, KMMLU-Redux 53.0

For Kanana-1.5-8B, the comparison is:

• English: HRB 66.2, MCLM 60.5, KMMLU-Redux 64.0
• Korean: 67.2, 31.8, 53.4
• Language-Mixed: 74.6, 57.4, 64.4

These results support three conclusions emphasized in the paper: Korean-only CoT tends to damage reasoning-heavy performance, English-only CoT preserves reasoning better but is weaker on Korean-specific content, and Language-Mixed CoT is generally the best compromise. The authors further claim that this mixed format also contributes to cross-lingual transfer and multimodal gains, although the multimodal evidence is shown on another model rather than on KO-REAson-35B itself.

3. Data foundation: Yi-Sang and Yi-Sang-HQ

KO-REAson-35B is trained on Yi-Sang-HQ, a curated subset of the larger Yi-Sang Korean reasoning corpus. The full Yi-Sang resource consists of 5.79M native Korean prompts, 3.7M long reasoning traces, and 6.77B tokens overall. The final KO-REAson models do not use the entire 3.7M-trace set; they use the smaller, high-yield 260k Yi-Sang-HQ subset (Son et al., 5 Oct 2025).

The data pipeline begins with web collection from public Korean Q&A and community sites. The authors compiled 54 candidate sites and assigned each a legal category: A for crawling and redistribution permitted, B for crawling allowed but commercial use or redistribution prohibited, and C for crawling prohibited. They then excluded category C sites, low-volume sources, heavily obfuscated sites, and near-duplicate sources, removing 26 websites at that stage. Data from category B sites was used for training and analysis but not redistributed.

Prompt filtering is deliberately light. Prompts are kept verbatim, without template normalization or rewriting, in order to preserve typos, abbreviations, mixed script, and internet style. Filtering removes prompts with Korean-character ratio < 30%, length < 50 characters, or length > 8,192 characters. Because the web-collected pool lacked enough competition-style problems, additional prompts were taken from OpenThought and translated into Korean using Gemini-2.5-Flash. The paper notes that earlier translation attempts using GPT-4o-mini, Qwen2.5-72B-Instruct, and Gemini-2.0-Flash caused training instability.

The reasoning traces are not based on scraped web answers. Instead, the authors regenerate all targets from the prompt alone. They considered and rejected agreement sampling with multiple samples plus an LLM judge, as well as hint-based refinement using scraped answers, citing compute cost, leakage risk, artifact risk, distribution shift, and answer noisiness. The main teacher for trace generation is Qwen3-32B, and ablations show that Qwen3-32B with long Language-Mixed CoT is better than Qwen3-4B and better than short-CoT or solution-only supervision.

Yi-Sang-HQ is shaped by targeted filtering. The paper removes:

1. degeneration cases with endless repetition,
2. samples containing multiple > ... </think> blocks,

   1. samples whose final solution after `` is in a non-Korean language,

3. instances longer than 16k tokens,

4. benchmark-near-duplicates using an $n$-gram similarity filter with $n=13$ against held-in and held-out benchmarks.

The final 260k subset contains 62k OpenThought, 86k Code, 37k Science, and 66k Exams. The authors also apply two augmentations, mainly on exam-style data: style augmentation, which adds directives such as returning the final answer in \boxed{} format, and option augmentation, which uses BM25 retrieval over the exam pool to find similar questions and merge distractor options, subject to negation filtering, near-duplicate removal, a cap of 10 options, and preservation of the original correct option. The paper reports that both augmentations help, and neither dominates consistently.

4. Training recipe and implementation

KO-REAson-35B is initialized from A.X-3.1-35B, described as a LLaMA-style model family from SK Telecom, focused particularly on Korean, trained on approximately 2.1 trillion tokens, and strong on Korean benchmarks while still performing well in English (Son et al., 5 Oct 2025).

The model is trained by supervised fine-tuning only. The paper explicitly states several negative facts: there is no continued pretraining, no preference optimization, no DPO, no RLHF, no GRPO, no RLVR, and no explicit curriculum learning beyond the subset-selection process. The final supervision format is:

• Korean prompt
• language-mixed long CoT trace
• Korean final answer

All experiments use 5 epochs unless otherwise specified, but A.X-3.1-35B is trained for 3 epochs due to computational constraints. KO-REAson-35B therefore receives 3 epochs on Yi-Sang-HQ.

The reported training hyperparameters are:

• precision: bfloat16
• optimizer: AdamW
• learning rate: $2\times10^{-5}$
• warmup: 10%
• weight decay: $1\times10^{-5}$
• final global batch size: 512

The implementation uses PyTorch FSDP, Liger kernels, and FlashAttention-2. A packing ablation on smaller models found that packing improved speed but somewhat worsened benchmark performance; as a result, all reported models are trained without packing, including KO-REAson-35B.

The exact training sequence length is not specified. The paper does state that Yi-Sang-HQ drops traces longer than 16k tokens, and that evaluation uses max_tokens=32,768, but it does not give the training context limit for the 35B run. Hardware spans 8 NVIDIA H100 to 24 NVIDIA H200 GPUs, but the exact allocation or total compute specific to KO-REAson-35B is not reported.

5. Evaluation protocol and reported performance

Evaluation is conducted with vLLM using temperature $=0.7$, top_p $=0.9$, and max_tokens $=32,768$. Models are instructed to place the final answer in \boxed{...}, and scoring extracts the first valid boxed answer after the hidden think section; if no valid boxed answer appears, the item is counted wrong. Main results are reported as mean $\pm$ standard error over $n=3$ runs (Son et al., 5 Oct 2025).

The main Korean evaluation suite contains nine benchmarks: KMMLU-Redux, KMMLU-Pro, KMMLU-Hard, Math (Ko), AIME2024 (Ko), KSM, HAE-RAE Bench (HRB), CLIcK, and KoBALT-700. KO-REAson-35B’s reported scores are:

Benchmark	Score	Rank
KMMLU-Redux	$n$0	1st
KMMLU-Pro	$n$1	2nd
KMMLU-Hard	$n$2	1st
Math (Ko)	$n$3	1st
AIME2024 (Ko)	$n$4	2nd
KSM	$n$5	effectively 2nd/tied for 2nd
HRB	$n$6	1st
CLIcK	$n$7	tied 1st
KoBALT-700	$n$8	2nd

Its overall average is $n$9, the best among the compared systems. The strongest baseline average is QwQ-32B at $n=13$0, so KO-REAson-35B exceeds it by 4.4 points. Other reported baseline averages are GPT-OSS-20B: $n=13$1, DeepSeek-R1-32B: $n=13$2, and EXAONE-Deep-32B: $n=13$3.

Comparison against its own backbone reveals the model’s specialization profile. Relative to A.X-3.1-35B, KO-REAson-35B improves substantially on reasoning-heavy tasks: KMMLU-Hard rises from $n=13$4 to $n=13$5, Math from $n=13$6 to $n=13$7, AIME24 from $n=13$8 to $n=13$9, and KSM from $2\times10^{-5}$0 to $2\times10^{-5}$1. At the same time, the base model remains stronger on some Korean cultural or linguistic benchmarks: HRB falls from $2\times10^{-5}$2 to $2\times10^{-5}$3, CLIcK from $2\times10^{-5}$4 to $2\times10^{-5}$5, and KoBALT from $2\times10^{-5}$6 to $2\times10^{-5}$7. This makes KO-REAson-35B a reasoning-optimized specialization rather than a uniform improvement over the backbone.

The paper also reports cross-lingual evaluation on English benchmarks. Despite training on Korean prompts only, KO-REAson-35B improves over A.X-3.1-35B from AIME24 $2\times10^{-5}$8 to $2\times10^{-5}$9, from AIME25 $1\times10^{-5}$0 to $1\times10^{-5}$1, and from GPQA $1\times10^{-5}$2 to $1\times10^{-5}$3. The authors attribute this to universal math and science knowledge transfer and to the retention of English within Language-Mixed CoT. By contrast, the multimodal experiments in the paper are performed on Gemma-3-12B / KO-REAson-12B, not on the 35B model.

6. Interpretive significance, caveats, and release

KO-REAson-35B is most significant as evidence that a Korean reasoning model can reach top-tier benchmark performance through curated native-language data, long CoT distillation, and Language-Mixed CoT, without RL-based post-training. The paper’s broader empirical claim is that similar gains hold across the full 4B–35B model range, with an average improvement of $1\times10^{-5}$4 points across nine benchmarks for smaller and mid-sized models (Son et al., 5 Oct 2025).

Several common misconceptions are directly contradicted by the training setup. First, KO-REAson-35B is not a Korean-only reasoning model: its hidden trace distribution is intentionally mixed, with English functioning as the anchor language. Second, it is not an RL-trained model: the pipeline is explicitly SFT-only. Third, it is not the paper’s multimodal flagship: the multimodal analysis is demonstrated on KO-REAson-12B rather than on the 35B system.

The limitations are also explicit. On AIME2024 (Ko) and KSM, KO-REAson-35B remains second rather than first, and the paper attributes this partly to limited competition-style data, with only about 60k filtered translated OpenThought items retained in the final mixture. The pipeline depends heavily on Qwen3-32B as the trace generator. The subset-construction loop repeatedly used held-in benchmarks, which the authors acknowledge introduces some contamination risk. Exact training sequence length, exact 35B hardware allocation, and total compute are not specified. The paper also provides no direct human evaluation, relying instead on benchmark-based evidence.

Release details are clear. The authors state that they release the data-curation pipeline, evaluation system, datasets, and models at https://huggingface.co/KOREAson. KO-REAson-35B / KO-REAson-AX3_1-35B-1002 is released under Apache 2.0, while Yi-Sang and Yi-Sang-HQ are released under the MIT License. Some crawled data originated from category B sites and was used for training and analysis but was not redistributed.

In aggregate, KO-REAson-35B is best understood as a reasoning-specialized Korean post-trained model whose main contribution is methodological rather than architectural: it shows that native Korean prompts, teacher-generated long reasoning traces, and Language-Mixed CoT can produce a strong open 35B reasoning model with both Korean benchmark leadership and nontrivial English transfer, while also exposing the tradeoff between reasoning gains and some aspects of culturally or linguistically specialized Korean performance.