XCoT: Cross-lingual Chain-of-Thought

• Cross-lingual Chain-of-Thought (XCoT) is a framework where models employ different languages for input, intermediate reasoning, and output to tackle multilingual challenges.
• It includes diverse methods such as English pivoting, multilingual collaborative reasoning, and latent-space alignment, each balancing transfer benefits and linguistic nuances.
• Empirical results from benchmarks like MGSM and XCOPA show that XCoT improves reasoning accuracy and efficiency while mitigating issues like overthinking and hallucination.

Searching arXiv for papers on Cross-lingual Chain-of-Thought to ground the article. arxiv_search(query="cross-lingual chain-of-thought multilingual reasoning", max_results=10) arxiv_search(query="XCoT cross-lingual chain-of-thought", max_results=10) Cross-lingual Chain-of-Thought (XCoT) is a family of multilingual reasoning settings and methods in which a model’s prompt language, intermediate reasoning language, and answer language are allowed to differ. Across the literature, the term does not denote a single algorithm. Instead, it covers prompt-time pivot-language reasoning, multilingual self-consistency and collaborative reasoning, instruction-tuned cross-lingual transfer, language-mixed reasoning traces, adaptive routing through “thinking languages,” and more recent latent-space or alignment-based variants that reduce reliance on surface-form tokens (Shi et al., 2022, Qin et al., 2023, Chai et al., 2024, Son et al., 5 Oct 2025, Bashir et al., 9 Mar 2026).

1. Conceptual scope and variants

The XCoT literature is best understood as a spectrum of design choices for where reasoning occurs and how languages interact during inference or training. A survey of Chain-of-X methods does not define XCoT as a standalone named paradigm, but its general CoX formalism naturally accommodates language-bridging intermediate nodes, multilingual dictionary augmentations, and cross-lingual reasoning chains (Xia et al., 2024).

Several distinct formulations recur in the literature.

Formulation	Core mechanism	Representative papers
Pivot-language CoT	Non-English question, reasoning in English, answer in English or the target language	(Shi et al., 2022, Qin et al., 2023)
Multilingual collaborative reasoning	Multiple language-specific CoT paths are compared, refined, or voted over	(Ranaldi et al., 2023, 2406.13940, Yu et al., 2 Apr 2025)
Cross-lingual instruction tuning	Queries are translated or code-switched during SFT; CoT is supervised in a high-resource or mixed language	(Chai et al., 2024, Upadhayay et al., 2023, Zheng et al., 17 Jul 2025)
Language-mixed or adaptive CoT	Reasoning code-switches between English and a target language, or routing selects a “thinking language” per query	(Son et al., 5 Oct 2025, Huang et al., 27 Jan 2025)
Latent or alignment-based XCoT	Reasoning is compressed into continuous latents or aligned in soft-token / logic space across languages	(Bashir et al., 9 Mar 2026, Zhang et al., 22 Apr 2026, He et al., 25 Jun 2026)

This plurality matters because empirical conclusions about “whether XCoT helps” depend strongly on which formulation is used. Prompt-time English pivots, multilingual voting, mixed-language reasoning, and latent-space alignment solve different failure modes and impose different costs.

2. Benchmarks, emergence, and early prompting-based XCoT

A foundational empirical result is that multilingual CoT reasoning emerges with scale. The MGSM benchmark introduced 250 GSM8K problems manually translated into ten languages and showed that large models can solve non-English grade-school math with explicit reasoning, including underrepresented languages such as Bengali and Swahili (Shi et al., 2022). On MGSM with PaLM-540B and native exemplars, direct prompting reached 18.6 average accuracy, Native-CoT reached 48.1, EN-CoT reached 51.3, and Translate-EN reached 55.0, establishing two early XCoT findings: explicit CoT is much better than direct answering, and English reasoning can equal or outperform same-language reasoning even when the question is non-English (Shi et al., 2022).

Subsequent prompt-only work converted this observation into explicit cross-lingual prompting schemes. Cross-lingual Prompting (CLP) splits inference into an alignment stage and a solver stage: the model first “understands the task in English step-by-step,” then solves it step-by-step in English, and Cross-lingual Self-consistent Prompting (CLSP) ensembles answers across languages by majority vote (Qin et al., 2023). On MGSM with GPT-3.5, En-CoT averaged 57.8, Translate-En 68.4, CLP 70.6, and CLSP 76.7; on XCOPA, CLP reached 85.3 and CLSP 92.7, close to the reported human score of 97.6 (Qin et al., 2023). This made explicit that cross-lingual alignment prompting is more than plain translation: the alignment stage can perform step-by-step translation, key information extraction, restatement, and even preliminary solution steps.

Cross-lingual Tree-of-Thoughts extended prompt-only XCoT from pivot-language reasoning to multilingual collaboration. Instead of a single English reasoning path, Cross-ToT asks several “experts” to reason in their mother tongues, compare intermediate steps, and iteratively converge to a definitive answer (Ranaldi et al., 2023). On MGSM with GPT-3.5, Cross-ToT reached 80.6 average accuracy versus 76.6 for Cross-CoT, and on XCOPA it reached 93.6, outperforming previous multilingual prompting baselines while reducing the number of interactions (Ranaldi et al., 2023). AutoCAP retained the multilingual-ensemble view but automated two decisions that earlier systems left fixed: which languages to use and how much weight each language’s reasoning path should receive (2406.13940). On MGSM with GPT-3.5, AutoCAP reached 78.6 average accuracy versus 75.5 for CLSP, with ablations showing that automatic language selection contributed more than automatic weight allocation, though both mattered (2406.13940).

The same ensemble logic was later reframed as Cross-Lingual Consistency (CLC), where multiple reasoning paths are sampled in multiple languages and final answers are aggregated by majority vote (Yu et al., 2 Apr 2025). On MGSM with Gemma2-9B-Instruct, exhaustive evaluation over all 2,047 non-empty subsets of 11 languages showed that the best six-language combination reached 92.09 average accuracy, while using all 11 languages reached 91.26; relative to monolingual self-consistency, the reported gains ranged from 4.1 to 18.5 percentage points (Yu et al., 2 Apr 2025). A consistent theme across these prompting-based methods is that cross-lingual diversity helps until added languages begin to inject more noise than complementary evidence.

3. Training-time XCoT and cross-lingual knowledge transfer

Prompting alone does not align multilingual representations. A second strand therefore treats XCoT as a supervised transfer problem. The xCoT framework builds multilingual instruction data by translating GSM8K queries into ten non-English languages while keeping the responses in English, then augments instruction tuning with code-switched in-context examples, Random Online CoT, and cross-lingual distillation (Chai et al., 2024). On MGSM, xCoT raised Llama-2-7B to 47.7 average accuracy and Llama-2-13B to 51.5, exceeding comparable multilingual math tuning baselines (Chai et al., 2024). The central design assumption is that English is a high-resource reasoning pivot and that non-English queries should be aligned to that English reasoning space during training, not only at inference.

TaCo adopts an even more explicit translation-assisted chain. During instruction tuning, a non-English prompt is answered by first translating the instruction into English, then producing an English answer, then translating the answer back into the target language (Upadhayay et al., 2023). This turns translation itself into part of the supervised reasoning trajectory. On the multilingual Vicuna Benchmark, the Nepali TaCo model reached an 88.1 average GPT-4 judged score versus 39.3 for standard Nepali instruction tuning, and Persian TaCo reached 83.9 versus 46.5 for Persian instruction tuning (Upadhayay et al., 2023). The method is explicitly motivated as a low-cost way to transfer the reasoning abilities of an English-centered model to low-resource languages without full multilingual pretraining.

More recent training-time variants no longer assume a fixed English pivot. AdaCoT treats “thinking language” as a learned routing decision over English, Chinese, Indonesian, and the target language itself, selecting direct generation or cross-lingual CoT pathways according to reward-model scores (Huang et al., 27 Jan 2025). Its reported gains on multilingual factual reasoning and cross-lingual consistency are strongest in low-resource settings, and on mTruthfulQA it improves 30 out of 32 languages relative to the base LLaMA3.1-8B model (Huang et al., 27 Jan 2025). Language-Mixed CoT takes a different view: instead of selecting one reasoning language, it supervises code-switched traces in which Korean prompts are answered with English-dominant reasoning containing preserved Korean spans, followed by a Korean final answer (Son et al., 5 Oct 2025). The resulting KO-REAson-35B reached $64.0_{\pm 2.5}$ overall average across nine Korean benchmarks, ranked first on 5/9 and second on the rest, and the family-wide average improvement over baselines was +18.6 points (Son et al., 5 Oct 2025).

A further extension uses XCoT explicitly for hallucination control rather than raw accuracy. CCL-XCoT combines curriculum-based contrastive alignment during continued pretraining with XCoT-style instruction tuning in which a low-resource question is answered via English reasoning and then rendered in the target language (Zheng et al., 17 Jul 2025). The reported effect is a reduction in hallucination rates by up to 62% and large gains in hallucination-free generation for Chinese, Malay, and Tamil without external retrieval (Zheng et al., 17 Jul 2025). In parallel, SOLAR aligns soft-token summaries of multilingual reasoning traces to an English pivot during supervised fine-tuning, improving accuracy by up to +17.7 over the base model and by up to +3.8 over standard SFT, with the largest gains on low-resource languages (He et al., 25 Jun 2026). These approaches treat XCoT less as a prompting heuristic than as an architectural or training prior about where reliable reasoning lives in multilingual models.

4. What multilingual reasoning traces reveal

One of the clearest empirical lessons from recent work is that the usefulness of CoT is highly conditional on language and model capacity. A large-scale narrative comprehension study operationalized XCoT as an experimental setting rather than a new algorithm: the same monolingual next-sentence-prediction task was run in English, Swahili, and Hausa, while models were either asked to answer directly or to “explain your process step by step” (Chavan et al., 29 Oct 2025). On 10,000 questions per language, direct-answer accuracy was 81.61/78.35/76.02 for GPT-4 Turbo, 80.79/77.13/75.04 for Gemini 1.5 Flash, and 80.71/68.71/59.43 for LLaMA 3 70B across English/Swahili/Hausa. On a matched 1,000-question CoT subset, CoT helped LLaMA 3 in Swahili and Hausa by +4.6 and +4.8 points, but hurt GPT-4 and Gemini in most non-English settings (Chavan et al., 29 Oct 2025). The paper attributes these failures to “overthinking,” implicit mistranslation, and the reframing of a constrained comprehension task as open-ended narrative continuation.

A broader multilingual CoT evaluation extends this point from accuracy to trace-level behavior. It studies language compliance, answer consistency, cross-lingual interchangeability of thinking traces, and perturbation-based faithfulness across MMMLU and MGSM (Zhao et al., 10 Oct 2025). Explicit instruction often fails to force the reasoning trace into the requested low-resource language, whereas prompt hacking greatly increases trace-language compliance; yet higher compliance is frequently accompanied by lower answer accuracy, especially in weaker languages (Zhao et al., 10 Oct 2025). The same work also shows that the quality of a thinking trace depends on the prompt language in which it was produced and that models rely on these traces to varying degrees once truncation or error injection is applied. This suggests that XCoT involves at least three partially separable variables: prompt language, thinking-trace language, and final-answer language.

The older MGSM findings already hinted at this separation. EN-CoT frequently surpassed Native-CoT, and multilingual exemplars outperformed English-only exemplars even for target languages absent from the prompt (Shi et al., 2022). Recent trace-focused evaluations make the implication sharper: successful XCoT is not equivalent to generating more non-English rationale text. In some regimes, forcing the visible reasoning into the target language improves compliance but weakens reasoning quality because the model’s most reliable inferential patterns remain anchored in a different language.

5. From token-space reasoning to latent and efficient XCoT

A major recent development is the move away from explicit token-level CoT as the sole carrier of cross-lingual reasoning. Continuous CoT, implemented through the CODI framework, replaces natural-language reasoning traces with a short sequence of latent vectors between special thought tokens and trains this student path together with an explicit-CoT teacher (Bashir et al., 9 Mar 2026). Across English, Chinese, German, French, and Urdu on GSM8K-Aug-NL and CommonsenseQA, continuous CoT is especially strong in low-resource and zero-shot regimes. On CommonsenseQA with multilingual training excluding Urdu, zero-shot Urdu accuracy reached 35.95 for CODI versus 23.01 for CoT-SFT; with Urdu included, CODI reached 41.20 versus 34.73 for CoT-SFT (Bashir et al., 9 Mar 2026). At the same time, the continuous thought sequence compressed reasoning by roughly $29\times$ to $50\times$, supporting the hypothesis that latent reasoning spaces can be more language-invariant than token sequences.

Efficiency-oriented work pushes in a complementary direction. UL-XCoT constructs a unified logic space by projecting away language-specific components of hidden states, then selects a small per-query language set and prunes low-quality multilingual trajectories during decoding (Zhang et al., 22 Apr 2026). On PolyMath and MMLU-ProX-Lite with DeepSeek-R1-Distill-Qwen-7B, UL-XCoT achieved competitive or improved accuracy while cutting decoding cost sharply: on PolyMath it reduced token usage by over 50% versus AUTOCAP and over 65% versus self-consistency, and on MMLU-ProX-Lite it improved average accuracy from 40.5 for CLSP to 43.6 while reducing average tokens from 27,679.3 to 10,543.6 (Zhang et al., 22 Apr 2026). This reframes XCoT from brute-force multilingual voting to geometry-aware allocation of inference budget.

An adjacent line shows that cross-lingual reasoning can also be induced without explicit multilingual CoT traces. MLPrompt identifies an error-prone rule inside a long English prompt, rewrites that rule in another language, and regenerates the output (Wang et al., 2024). On text-to-MIP generation with GPT-4o, the baseline score of 0.472 increased to 0.844 when the problematic rule was replaced by its Mandarin version, outperforming CoT, ToT, and self-consistency on that task (Wang et al., 2024). The paper does not present MLPrompt as XCoT proper, but it provides evidence that cross-lingual signals can function as reasoning control variables even when no explicit multilingual rationale is generated.

6. Persistent tensions and open directions

The XCoT literature is unified less by one method than by a recurring tension: multilingual reasoning benefits from cross-lingual transfer, yet the form that transfer should take remains unsettled. English as a reasoning pivot is effective in early prompting work, in translation-assisted tuning, in adaptive routing, in hallucination mitigation, and in soft-token alignment (Shi et al., 2022, Upadhayay et al., 2023, Huang et al., 27 Jan 2025, Zheng et al., 17 Jul 2025, He et al., 25 Jun 2026). At the same time, several papers document the liabilities of explicit English pivoting: translation bottlenecks, semantic drift, cultural misreadings, overthinking, and loss of target-language nuance (Bashir et al., 9 Mar 2026, Chavan et al., 29 Oct 2025). A plausible implication is that the field is moving from explicit English reasoning toward more abstract cross-lingual substrates—mixed-language traces, aligned soft-token spaces, continuous latents, or unified logic spaces—while retaining English as an anchor only where it is empirically useful.

A second unresolved issue is selection: which languages should participate in XCoT, and how many. AutoCAP learns language choice and weighting through prompting, UL-XCoT chooses languages in a projected logic space, and CLC shows empirically that a six-language ensemble can outperform both smaller and larger sets (2406.13940, Zhang et al., 22 Apr 2026, Yu et al., 2 Apr 2025). This suggests that multilingual diversity has diminishing returns and that language choice is query- and model-dependent rather than fixed.

A third issue is faithfulness. Multilingual CoT traces are not merely translations of one another, and high answer accuracy does not guarantee that a visible rationale is causally responsible for the answer (Zhao et al., 10 Oct 2025). This concern is amplified in low-resource settings, where visible rationales can reflect mistranslation or hallucinated grounding rather than genuine inference (Chavan et al., 29 Oct 2025). Consequently, future XCoT evaluation is likely to require joint measurement of final-answer performance, cross-language consistency, and perturbation-based faithfulness rather than accuracy alone.

Finally, domain coverage remains limited. Much of the literature concentrates on arithmetic reasoning, multiple-choice knowledge, or narrow narrative comprehension; several papers explicitly note that technical documents, law, medicine, dialogue, and broader genre variation remain underexplored (Chavan et al., 29 Oct 2025, Bashir et al., 9 Mar 2026). The present state of XCoT therefore supports a precise but bounded conclusion: multilingual reasoning improves when language is treated as a controllable variable inside the reasoning process itself, but the most effective control mechanism varies sharply across models, languages, domains, and resource regimes.