BridgeX-ICL: Neuron-Guided Cross-Lingual Transfer

• BridgeX-ICL is a neuron-guided method that uses high-activation overlap neurons to select an effective bridge language for zero-shot cross-lingual in-context learning.
• The method constructs language-specific neuron sets and uses cosine and HSIC metrics to quantify similarity between source–target and candidate bridge languages.
• Empirical results show significant improvements in bilingual lexicon induction and machine reading comprehension, often rivaling few-shot approaches without any fine-tuning.

BridgeX-ICL is a neuron-guided method for improving zero-shot cross-lingual in-context learning (X-ICL) on low-resource languages without fine-tuning. Its central idea is that multilingual transfer can be improved by identifying language overlap neurons—feed-forward neurons that are strongly activated by both languages in a source–target pair—and then using those overlap patterns to select an effective bridge language for prompting. In this formulation, the “bridge” is both external and internal: externally, it is a language inserted into the prompt; internally, it is the shared neuron subspace through which cross-lingual transfer is presumed to occur (Xu et al., 23 Aug 2025).

1. Problem setting and motivation

BridgeX-ICL is formulated for zero-shot X-ICL: given a source language $L_s$ and a low-resource target language $L_t$, a multilingual LLM must solve a task in $L_t$ at inference time with no parameter updates. The method is motivated by the practical difficulty of improving low-resource-language performance when multilingual pretraining is skewed toward high-resource languages and when fine-tuning is costly or infeasible. The paper explicitly frames this as a “language-bridge” problem: a suitable bridge language $L_y$ may improve transfer from $L_s$ to $L_t$ (Xu et al., 23 Aug 2025).

The motivating example is concrete. In Arabic $\rightarrow$ Hebrew bilingual lexicon induction (BLI), LLaMA 3 obtains 47.0% zero-shot accuracy, but specifying English as a bridge raises performance to 64.5%, even exceeding two-shot X-ICL. This motivates the central operational question: not whether a bridge language can help, but how to select the best bridge language for a given source–target pair.

The method is positioned against two existing tendencies in the literature. Prior X-ICL work has concentrated mainly on example selection or alignment rather than on bridge-language selection, while prior multilingual neuron studies have focused on language-specific neurons. BridgeX-ICL instead asks whether shared neurons across languages, rather than language-specific ones, are the relevant substrate for cross-lingual transfer. The paper’s core hypothesis is therefore explicit: shared / overlap neurons across languages can facilitate cross-lingual transfer, especially for low-resource languages (Xu et al., 23 Aug 2025).

2. Overlap neurons and the internal linguistic spectrum

BridgeX-ICL assumes that language-relevant neurons are concentrated in FFN layers. For the $i$-th layer, the paper writes

$$\boldsymbol{h}_i = \sigma (\tilde{\boldsymbol{h}_i} \boldsymbol{W}_1^i ) \cdot \boldsymbol{W}_2^i$$

with neuron $j$ defined as one column of $L_t$0. The activation value of neuron $L_t$1 is

$L_t$2

If this value is $L_t$3, the neuron is counted as activated (Xu et al., 23 Aug 2025).

For each language $L_t$4, the method constructs a language-associated neuron set $L_t$5 by ranking neurons by activation frequency and selecting the top $L_t$6 neurons. For two languages $L_t$7 and $L_t$8, the overlap neurons are defined layerwise as

$L_t$9

These language overlap neurons are thus the neurons that lie in the high-activation sets of both languages. On this overlap set, the paper defines a cosine-style similarity: $L_t$0 where $L_t$1 and $L_t$2 are activation-frequency vectors restricted to $L_t$3.

This construction underlies what the paper calls the model’s internal linguistic spectrum. The proposed interpretation is mechanistic: overlap neurons encode shared processing pathways between languages and may capture shared semantic information even across families. The paper further associates middle layers with semantic understanding and final layers with generation or language coding for next-token prediction. Its logit-lens analysis suggests that during translation, latent states in middle layers move toward representations associated with a high-resource language such as English. This supports the claim that a useful bridge language is not simply a genealogically related language, but a language whose internal neuron behavior matches the source–target overlap structure in the model (Xu et al., 23 Aug 2025).

The resulting linguistic spectrum is only partly human-like. Heatmaps partly align with language genealogy within families, but not uniformly across families. The paper highlights the example that Arabic can be slightly more similar to French than to Hebrew in LLaMA 3’s overlap-neuron space, which it attributes plausibly to training-distribution effects rather than to linguistic genealogy.

3. HSIC-based bridge selection and the BridgeX-ICL pipeline

BridgeX-ICL uses probe data derived from ground-truth bilingual lexicons to ensure what it calls “full activation of these anchored neurons.” The method uses MUSE bilingual dictionaries for bridge-selection probing and FLORES+ for language-neuron identification. For each language pair $L_t$4, it collects $L_t$5 word pairs that the LLM can translate accurately, prompts the model in both directions, and records FFN activations during generation rather than passive token reading. The stated purpose is to make the neuron signal reflect validated bilingual competence rather than noise (Xu et al., 23 Aug 2025).

Given source $L_t$6 and target $L_t$7, let $L_t$8 be their overlap neurons. Using the bilingual probes, the method constructs an activation matrix

$L_t$9

for overlap neurons. For a candidate bridge language $L_y$0, it constructs

$L_y$1

where the paper states that $L_y$2 contains neurons specific to bridge language $L_y$3, excluding neurons already in the source–target overlap set and excluding neurons shared with another bridge language $L_y$4. After average pooling so that $L_y$5 and $L_y$6 have the same row dimension $L_y$7, dependence is measured with HSIC: $L_y$8 where $L_y$9.

Rather than using only whole-matrix HSIC, the paper adopts a bidirectional maximum matching strategy: $L_s$0 and then averages over selected layers: $L_s$1 The chosen bridge is

$L_s$2

Layer selection is itself data-driven. Using embedding semantic similarity across layers, the method selects stable middle-layer regions: layers 10–21 for LLaMA 3 and layers 15–23 for Mistral. End to end, the pipeline is: prepare bilingual resources; build probe data; identify language neuron sets $L_s$3; construct source–target overlap neurons $L_s$4; compute bridge-specific neuron sets; evaluate candidate bridges with the HSIC score; choose $L_s$5; and finally prompt the downstream task in a zero-shot bridge format using the selected bridge language (Xu et al., 23 Aug 2025).

4. Experimental design and evaluation setting

The empirical study covers two cross-lingual tasks: Bilingual Lexicon Induction (BLI) and Machine Reading Comprehension (MRC) using Belebele. The evaluated models are LLaMA-3-8B and Mistral-7B-Instruct-v0.3. The language inventory contains 15 languages from 7 families: Indo-European (En, De, Fr, It, Pt, Es), Uralic (Fi, Hu), Afro-Asiatic (Ar, He), Austronesian (Id, Tl), Sino-Tibetan (Zh), Japonic (Ja), and Niger-Congo (Sw). The designated low-resource targets are Hebrew, Tagalog, Swahili, and Japanese (Xu et al., 23 Aug 2025).

The paper evaluates 15 source–target pairs: Zh-Ja, Zh-He, Zh-Tl, Zh-Sw, Ar-Ja, Ar-He, Ar-Tl, Ar-Sw, Id-Ja, Id-He, Id-Tl, Id-Sw, En-He, En-Tl, En-Sw. These include both high-low and moderate-low settings. For dictionaries, the method uses MUSE where available and constructs missing pairs through English pivoting; for Swahili, it uses wiktionary_bli to build En-Sw. Word pairs are validated with both Google and Microsoft translators, and 1,000 consistent pairs are kept per language pair.

Bridge candidates are restricted to six well-supported Indo-European languages: En, De, Fr, It, Pt, Es. Metrics are Precision@1 for BLI and multiple-choice accuracy for MRC. Baselines comprise Zero-shot, Few-shot (1, 2, 3, 4 shots), and Zero-shot with bridge. Bridge-selection baselines include Human source, Human target, English bridge, Sharing matter, IoU score, and LAPE$L_s$6. The IoU baseline is defined as

$L_s$7

This design isolates the contribution of neuron-guided bridge selection rather than few-shot example accumulation (Xu et al., 23 Aug 2025).

5. Quantitative findings

The paper reports average improvements of BridgeX-ICL over zero-shot baselines of +6.02% and +5.25% across the two tasks; in the task-wise discussion it reports +6.03% on MRC for LLaMA 3 and +4.48% for Mistral. On BLI, BridgeX-ICL often reaches performance comparable to two-shot X-ICL while remaining zero-shot (Xu et al., 23 Aug 2025).

The strongest gains appear on several BLI pairs. For LLaMA 3, BridgeX-ICL improves over zero-shot by +10.80 on Zh-Ja, +12.60 on Zh-He, +11.40 on Zh-Tl, +10.70 on Ar-Ja, +17.50 on Ar-He, +16.60 on Id-He, and +14.90 on En-He. For Mistral, reported improvements include +7.90 on Zh-Ja, +8.60 on Zh-He, +6.40 on Zh-Tl, +8.90 on Ar-Ja, and +8.60 on Ar-Tl. The method is not uniformly positive: the paper also reports negative cases, including En-Tl: -1.30 and En-Sw: -2.60 for LLaMA 3, and Id-He: -3.30 and En-He: -0.10 for Mistral.

Few-shot prompting is described as unstable, especially for Mistral: one-shot can be worse than zero-shot, gains tend to saturate by 3 shots, and additional examples beyond 3 often do not help. This is one of the paper’s practical arguments for bridge prompting as a cheaper intervention than few-shot demonstration scaling.

Bridge selection itself is model-specific. English is selected as the optimal bridge in 9/15 pairs, but not universally; Es, Fr, Pt, and De are also selected, and the preferred bridge differs across LLaMA 3 and Mistral. Human-genealogy-based bridge choices perform worst overall. The appendix comparison between HSIC and cosine similarity further supports the proposed metric: for example, on Zh-Ja with LLaMA 3, HSIC selects En and yields 77.90, whereas cosine selects De and yields 76.60; on Id-Tl, HSIC selects Fr and yields 61.00, whereas cosine selects De and yields 57.00.

A key ablation concerns probe construction. Replacing the MUSE-based activated bilingual probes with simpler FLORES+-derived bilingual tokens generally reduces performance. Reported examples include Ar-Tl: 52.00 $L_s$8 56.80 and En-He: 66.60 $L_s$9 71.80 for LLaMA 3, and Id-Ja: 50.80 $L_t$0 58.00 and Id-He: 33.40 $L_t$1 38.40 for Mistral. This is the main empirical support for the claim that validated bilingual generation better identifies useful overlap neurons (Xu et al., 23 Aug 2025).

6. Interpretation, limitations, and place within ICL research

BridgeX-ICL advances a specific view of multilingual transfer: low-resource transfer is mediated not only by language-specific subnetworks but also by shared multilingual circuitry. The method operationalizes this by turning overlap-neuron geometry into a bridge-language selector, thereby making bridge prompting depend on the model’s internal linguistic spectrum rather than on external language-family heuristics. A plausible implication is that cross-lingual prompting should be treated as a model-internal alignment problem as much as a linguistic one (Xu et al., 23 Aug 2025).

The paper also states clear limitations. It evaluates only 15 language pairs and 4 low-resource targets; the inferred linguistic distance is probe-induced and qualitative, not a precise universal metric; the method depends on high-quality probe data and on the target language being present enough in the LLM to yield meaningful activations; and candidate bridges are limited to well-supported Indo-European languages. It further notes a substantive failure mode: bridge prompting can hurt when the selected bridge does not match the model’s actual transfer path or when the source–target pair is already well handled.

Within the broader ICL literature, BridgeX-ICL belongs to a family of methods that modify inference by exploiting internal or structural information rather than by standard fine-tuning. Adjacent work has aligned supervised fine-tuning with ICL through internal activation matching in IA2 (Mishra et al., 26 Sep 2025), reduced prompt-template arbitrariness through reusable soft-token schemas in ICL Markup (Brunet et al., 2023), probed inference-time learning via reversible and non-reversible substitution ciphers in ICL CIPHERS (Fang et al., 28 Apr 2025), and improved fine-grained emotion recognition by replacing semantic retrieval with task-specific prototype selection and exclusionary candidate pruning in E-ICL (Ren et al., 2024). BridgeX-ICL is distinct within this landscape because it does not alter parameters or prompt structure directly; instead, it uses overlap neurons and HSIC-based bridge selection to decide how a multilingual prompt should be framed.

In that sense, BridgeX-ICL is both a practical zero-shot prompting method and a mechanistic claim about multilingual LLMs. Its central assertion is that the best bridge language is the one that best matches the model’s own shared neuron structure for a source–target pair, not necessarily the one closest under human linguistic taxonomy.