---
title: 'XCoT: Cross-lingual Chain-of-Thought'
url: https://www.emergentmind.com/topics/cross-lingual-chain-of-thought-xcot
type: topic
---

# XCoT: Cross-lingual Chain-of-Thought

Searching arXiv for recent papers on Cross-lingual Chain-of-Thought to ground the article.
arxiv_search(query="cross-lingual chain-of-thought multilingual reasoning", max_results=10)
arxiv_search(query="XCoT cross-lingual chain-of-thought", max_results=10)
Cross-lingual Chain-of-Thought (XCoT) is a family of multilingual reasoning settings and methods in which a model’s prompt language, intermediate reasoning language, and answer language are allowed to differ. Across the literature, the term does not denote a single algorithm. Instead, it covers prompt-time pivot-language reasoning, multilingual self-consistency and collaborative reasoning, instruction-tuned cross-lingual transfer, language-mixed reasoning traces, adaptive routing through “thinking languages,” and more recent latent-space or alignment-based variants that reduce reliance on surface-form tokens [2210.03057], [2310.14799], [2401.07037], [2510.04230], [2603.08177].

## 1. Conceptual scope and variants

The XCoT literature is best understood as a spectrum of design choices for where reasoning occurs and how languages interact during inference or training. A survey of Chain-of-X methods does not define XCoT as a standalone named paradigm, but its general CoX formalism naturally accommodates language-bridging intermediate nodes, multilingual dictionary augmentations, and cross-lingual reasoning chains [2404.15676].

Several distinct formulations recur in the literature.

| Formulation | Core mechanism | Representative papers |
|---|---|---|
| Pivot-language CoT | Non-English question, reasoning in English, answer in English or the target language | [2210.03057], [2310.14799] |
| Multilingual collaborative reasoning | Multiple language-specific CoT paths are compared, refined, or voted over | [2311.08097], [2406.13940], [2504.01857] |
| Cross-lingual instruction tuning | Queries are translated or code-switched during SFT; CoT is supervised in a high-resource or mixed language | [2401.07037], [2311.10797], [2507.14239] |
| Language-mixed or adaptive CoT | Reasoning code-switches between English and a target language, or routing selects a “thinking language” per query | [2510.04230], [2501.16154] |
| Latent or alignment-based XCoT | Reasoning is compressed into continuous latents or aligned in soft-token / logic space across languages | [2603.08177], [2604.20090], [2606.26466] |

This plurality matters because empirical conclusions about “whether XCoT helps” depend strongly on which formulation is used. Prompt-time English pivots, multilingual voting, mixed-language reasoning, and latent-space alignment solve different failure modes and impose different costs.

## 2. Benchmarks, emergence, and early prompting-based XCoT

A foundational empirical result is that multilingual CoT reasoning emerges with scale. The MGSM benchmark introduced 250 GSM8K problems manually translated into ten languages and showed that large models can solve non-English grade-school math with explicit reasoning, including underrepresented languages such as Bengali and Swahili [2210.03057]. On MGSM with PaLM-540B and native exemplars, direct prompting reached 18.6 average accuracy, Native-CoT reached 48.1, EN-CoT reached 51.3, and Translate-EN reached 55.0, establishing two early XCoT findings: explicit CoT is much better than direct answering, and English reasoning can equal or outperform same-language reasoning even when the question is non-English [2210.03057].

Subsequent prompt-only work converted this observation into explicit cross-lingual prompting schemes. Cross-lingual Prompting (CLP) splits inference into an alignment stage and a solver stage: the model first “understands the task in English step-by-step,” then solves it step-by-step in English, and Cross-lingual Self-consistent Prompting (CLSP) ensembles answers across languages by majority vote [2310.14799]. On MGSM with GPT-3.5, En-CoT averaged 57.8, Translate-En 68.4, CLP 70.6, and CLSP 76.7; on XCOPA, CLP reached 85.3 and CLSP 92.7, close to the reported human score of 97.6 [2310.14799]. This made explicit that cross-lingual alignment prompting is more than plain translation: the alignment stage can perform step-by-step translation, key information extraction, restatement, and even preliminary solution steps.

Cross-lingual Tree-of-Thoughts extended prompt-only XCoT from pivot-language reasoning to multilingual collaboration. Instead of a single English reasoning path, Cross-ToT asks several “experts” to reason in their mother tongues, compare intermediate steps, and iteratively converge to a definitive answer [2311.08097]. On MGSM with GPT-3.5, Cross-ToT reached 80.6 average accuracy versus 76.6 for Cross-CoT, and on XCOPA it reached 93.6, outperforming previous multilingual prompting baselines while reducing the number of interactions [2311.08097]. AutoCAP retained the multilingual-ensemble view but automated two decisions that earlier systems left fixed: which languages to use and how much weight each language’s reasoning path should receive [2406.13940]. On MGSM with GPT-3.5, AutoCAP reached 78.6 average accuracy versus 75.5 for CLSP, with ablations showing that automatic language selection contributed more than automatic weight allocation, though both mattered [2406.13940].

The same ensemble logic was later reframed as Cross-Lingual Consistency (CLC), where multiple reasoning paths are sampled in multiple languages and final answers are aggregated by majority vote [2504.01857]. On MGSM with Gemma2-9B-Instruct, exhaustive evaluation over all 2,047 non-empty subsets of 11 languages showed that the best six-language combination reached 92.09 average accuracy, while using all 11 languages reached 91.26; relative to monolingual self-consistency, the reported gains ranged from 4.1 to 18.5 percentage points [2504.01857]. A consistent theme across these prompting-based methods is that cross-lingual diversity helps until added languages begin to inject more noise than complementary evidence.

## 3. Training-time XCoT and cross-lingual knowledge transfer

Prompting alone does not align multilingual representations. A second strand therefore treats XCoT as a supervised transfer problem. The xCoT framework builds multilingual instruction data by translating GSM8K queries into ten non-English languages while keeping the responses in English, then augments instruction tuning with code-switched in-context examples, Random Online CoT, and cross-lingual distillation [2401.07037]. On MGSM, xCoT raised Llama-2-7B to 47.7 average accuracy and Llama-2-13B to 51.5, exceeding comparable multilingual math tuning baselines [2401.07037]. The central design assumption is that English is a high-resource reasoning pivot and that non-English queries should be aligned to that English reasoning space during training, not only at inference.

TaCo adopts an even more explicit translation-assisted chain. During instruction tuning, a non-English prompt is answered by first translating the instruction into English, then producing an English answer, then translating the answer back into the target language [2311.10797]. This turns translation itself into part of the supervised reasoning trajectory. On the multilingual Vicuna Benchmark, the Nepali TaCo model reached an 88.1 average GPT-4 judged score versus 39.3 for standard Nepali instruction tuning, and Persian TaCo reached 83.9 versus 46.5 for Persian instruction tuning [2311.10797]. The method is explicitly motivated as a low-cost way to transfer the reasoning abilities of an English-centered model to low-resource languages without full multilingual pretraining.

More recent training-time variants no longer assume a fixed English pivot. AdaCoT treats “thinking language” as a learned routing decision over English, Chinese, Indonesian, and the target language itself, selecting direct generation or cross-lingual CoT pathways according to reward-model scores [2501.16154]. Its reported gains on multilingual factual reasoning and cross-lingual consistency are strongest in low-resource settings, and on mTruthfulQA it improves 30 out of 32 languages relative to the base LLaMA3.1-8B model [2501.16154]. Language-Mixed CoT takes a different view: instead of selecting one reasoning language, it supervises code-switched traces in which Korean prompts are answered with English-dominant reasoning containing preserved Korean spans, followed by a Korean final answer [2510.04230]. The resulting KO-REAson-35B reached \(64.0_{\pm 2.5}\) overall average across nine Korean benchmarks, ranked first on 5/9 and second on the rest, and the family-wide average improvement over baselines was +18.6 points [2510.04230].

A further extension uses XCoT explicitly for hallucination control rather than raw accuracy. CCL-XCoT combines curriculum-based contrastive alignment during continued pretraining with XCoT-style instruction tuning in which a low-resource question is answered via English reasoning and then rendered in the target language [2507.14239]. The reported effect is a reduction in hallucination rates by up to 62% and large gains in hallucination-free generation for Chinese, Malay, and Tamil without external retrieval [2507.14239]. In parallel, SOLAR aligns soft-token summaries of multilingual reasoning traces to an English pivot during supervised fine-tuning, improving accuracy by up to +17.7 over the base model and by up to +3.8 over standard SFT, with the largest gains on low-resource languages [2606.26466]. These approaches treat XCoT less as a prompting heuristic than as an architectural or training prior about where reliable reasoning lives in multilingual models.

## 4. What multilingual reasoning traces reveal

One of the clearest empirical lessons from recent work is that the usefulness of CoT is highly conditional on language and model capacity. A large-scale narrative comprehension study operationalized XCoT as an experimental setting rather than a new algorithm: the same monolingual next-sentence-prediction task was run in English, Swahili, and Hausa, while models were either asked to answer directly or to “explain your process step by step” [2510.25187]. On 10,000 questions per language, direct-answer accuracy was 81.61/78.35/76.02 for GPT-4 Turbo, 80.79/77.13/75.04 for Gemini 1.5 Flash, and 80.71/68.71/59.43 for LLaMA 3 70B across English/Swahili/Hausa. On a matched 1,000-question CoT subset, CoT helped LLaMA 3 in Swahili and Hausa by +4.6 and +4.8 points, but hurt GPT-4 and Gemini in most non-English settings [2510.25187]. The paper attributes these failures to “overthinking,” implicit mistranslation, and the reframing of a constrained comprehension task as open-ended narrative continuation.

A broader multilingual CoT evaluation extends this point from accuracy to trace-level behavior. It studies language compliance, answer consistency, cross-lingual interchangeability of thinking traces, and perturbation-based faithfulness across MMMLU and MGSM [2510.09555]. Explicit instruction often fails to force the reasoning trace into the requested low-resource language, whereas prompt hacking greatly increases trace-language compliance; yet higher compliance is frequently accompanied by lower answer accuracy, especially in weaker languages [2510.09555]. The same work also shows that the quality of a thinking trace depends on the prompt language in which it was produced and that models rely on these traces to varying degrees once truncation or error injection is applied. This suggests that XCoT involves at least three partially separable variables: prompt language, thinking-trace language, and final-answer language.

The older MGSM findings already hinted at this separation. EN-CoT frequently surpassed Native-CoT, and multilingual exemplars outperformed English-only exemplars even for target languages absent from the prompt [2210.03057]. Recent trace-focused evaluations make the implication sharper: successful XCoT is not equivalent to generating more non-English rationale text. In some regimes, forcing the visible reasoning into the target language improves compliance but weakens reasoning quality because the model’s most reliable inferential patterns remain anchored in a different language.

## 5. From token-space reasoning to latent and efficient XCoT

A major recent development is the move away from explicit token-level CoT as the sole carrier of cross-lingual reasoning. Continuous CoT, implemented through the CODI framework, replaces natural-language reasoning traces with a short sequence of latent vectors between special thought tokens and trains this student path together with an explicit-CoT teacher [2603.08177]. Across English, Chinese, German, French, and Urdu on GSM8K-Aug-NL and CommonsenseQA, continuous CoT is especially strong in low-resource and zero-shot regimes. On CommonsenseQA with multilingual training excluding Urdu, zero-shot Urdu accuracy reached 35.95 for CODI versus 23.01 for CoT-SFT; with Urdu included, CODI reached 41.20 versus 34.73 for CoT-SFT [2603.08177]. At the same time, the continuous thought sequence compressed reasoning by roughly \(29\times\) to \(50\times\), supporting the hypothesis that latent reasoning spaces can be more language-invariant than token sequences.

Efficiency-oriented work pushes in a complementary direction. UL-XCoT constructs a unified logic space by projecting away language-specific components of hidden states, then selects a small per-query language set and prunes low-quality multilingual trajectories during decoding [2604.20090]. On PolyMath and MMLU-ProX-Lite with DeepSeek-R1-Distill-Qwen-7B, UL-XCoT achieved competitive or improved accuracy while cutting decoding cost sharply: on PolyMath it reduced token usage by over 50% versus AUTOCAP and over 65% versus self-consistency, and on MMLU-ProX-Lite it improved average accuracy from 40.5 for CLSP to 43.6 while reducing average tokens from 27,679.3 to 10,543.6 [2604.20090]. This reframes XCoT from brute-force multilingual voting to geometry-aware allocation of inference budget.

An adjacent line shows that cross-lingual reasoning can also be induced without explicit multilingual CoT traces. MLPrompt identifies an error-prone rule inside a long English prompt, rewrites that rule in another language, and regenerates the output [2409.11056]. On text-to-MIP generation with GPT-4o, the baseline score of 0.472 increased to 0.844 when the problematic rule was replaced by its Mandarin version, outperforming CoT, ToT, and self-consistency on that task [2409.11056]. The paper does not present MLPrompt as XCoT proper, but it provides evidence that cross-lingual signals can function as reasoning control variables even when no explicit multilingual rationale is generated.

## 6. Persistent tensions and open directions

The XCoT literature is unified less by one method than by a recurring tension: multilingual reasoning benefits from cross-lingual transfer, yet the form that transfer should take remains unsettled. English as a reasoning pivot is effective in early prompting work, in translation-assisted tuning, in adaptive routing, in hallucination mitigation, and in soft-token alignment [2210.03057], [2311.10797], [2501.16154], [2507.14239], [2606.26466]. At the same time, several papers document the liabilities of explicit English pivoting: translation bottlenecks, semantic drift, cultural misreadings, overthinking, and loss of target-language nuance [2603.08177], [2510.25187]. A plausible implication is that the field is moving from explicit English reasoning toward more abstract cross-lingual substrates—mixed-language traces, aligned soft-token spaces, continuous latents, or unified logic spaces—while retaining English as an anchor only where it is empirically useful.

A second unresolved issue is selection: which languages should participate in XCoT, and how many. AutoCAP learns language choice and weighting through prompting, UL-XCoT chooses languages in a projected logic space, and CLC shows empirically that a six-language ensemble can outperform both smaller and larger sets [2406.13940], [2604.20090], [2504.01857]. This suggests that multilingual diversity has diminishing returns and that language choice is query- and model-dependent rather than fixed.

A third issue is faithfulness. Multilingual CoT traces are not merely translations of one another, and high answer accuracy does not guarantee that a visible rationale is causally responsible for the answer [2510.09555]. This concern is amplified in low-resource settings, where visible rationales can reflect mistranslation or hallucinated grounding rather than genuine inference [2510.25187]. Consequently, future XCoT evaluation is likely to require joint measurement of final-answer performance, cross-language consistency, and perturbation-based faithfulness rather than accuracy alone.

Finally, domain coverage remains limited. Much of the literature concentrates on arithmetic reasoning, multiple-choice knowledge, or narrow narrative comprehension; several papers explicitly note that technical documents, law, medicine, dialogue, and broader genre variation remain underexplored [2510.25187], [2603.08177]. The present state of XCoT therefore supports a precise but bounded conclusion: multilingual reasoning improves when language is treated as a controllable variable inside the reasoning process itself, but the most effective control mechanism varies sharply across models, languages, domains, and resource regimes.

Source: https://www.emergentmind.com/topics/cross-lingual-chain-of-thought-xcot