---
title: 'BridgeX-ICL: Neuron-Guided Cross-Lingual Transfer'
url: https://www.emergentmind.com/topics/bridgex-icl
type: topic
---

# BridgeX-ICL: Neuron-Guided Cross-Lingual Transfer

BridgeX-ICL is a neuron-guided method for improving **zero-shot cross-lingual in-context learning (X-ICL)** on low-resource languages **without fine-tuning**. Its central idea is that multilingual transfer can be improved by identifying **language overlap neurons**—feed-forward neurons that are strongly activated by both languages in a source–target pair—and then using those overlap patterns to select an effective **bridge language** for prompting. In this formulation, the “bridge” is both external and internal: externally, it is a language inserted into the prompt; internally, it is the shared neuron subspace through which cross-lingual transfer is presumed to occur [2508.17078].

## 1. Problem setting and motivation

BridgeX-ICL is formulated for **zero-shot X-ICL**: given a source language \(L_s\) and a low-resource target language \(L_t\), a multilingual LLM must solve a task in \(L_t\) at inference time with no parameter updates. The method is motivated by the practical difficulty of improving low-resource-language performance when multilingual pretraining is skewed toward high-resource languages and when fine-tuning is costly or infeasible. The paper explicitly frames this as a “language-bridge” problem: a suitable bridge language \(L_y\) may improve transfer from \(L_s\) to \(L_t\) [2508.17078].

The motivating example is concrete. In **Arabic \(\rightarrow\) Hebrew** bilingual lexicon induction (BLI), **LLaMA 3** obtains **47.0%** zero-shot accuracy, but specifying **English as a bridge** raises performance to **64.5%**, even exceeding **two-shot X-ICL**. This motivates the central operational question: not whether a bridge language can help, but **how to select the best bridge language** for a given source–target pair.

The method is positioned against two existing tendencies in the literature. Prior X-ICL work has concentrated mainly on example selection or alignment rather than on bridge-language selection, while prior multilingual neuron studies have focused on **language-specific neurons**. BridgeX-ICL instead asks whether **shared neurons across languages**, rather than language-specific ones, are the relevant substrate for cross-lingual transfer. The paper’s core hypothesis is therefore explicit: **shared / overlap neurons across languages can facilitate cross-lingual transfer, especially for low-resource languages** [2508.17078].

## 2. Overlap neurons and the internal linguistic spectrum

BridgeX-ICL assumes that language-relevant neurons are concentrated in **FFN layers**. For the \(i\)-th layer, the paper writes
\[
\boldsymbol{h}_i = \sigma (\tilde{\boldsymbol{h}_i} \boldsymbol{W}_1^i ) \cdot \boldsymbol{W}_2^i
\]
with neuron \(j\) defined as one column of \(\boldsymbol{W}_1^i\). The activation value of neuron \(j\) is
\[
\sigma(\tilde{\boldsymbol{h}_i} \boldsymbol{W}_1^i)_j.
\]
If this value is \(>0\), the neuron is counted as activated [2508.17078].

For each language \(L_k\), the method constructs a language-associated neuron set \(\mathcal{T}_k\) by ranking neurons by **activation frequency** and selecting the top \(\tau N\) neurons. For two languages \(L_u\) and \(L_v\), the overlap neurons are defined layerwise as
\[
\mathcal{T}_{u,v}(i)=\mathcal{T}_u(i) \cap \mathcal{T}_v(i).
\]
These **language overlap neurons** are thus the neurons that lie in the high-activation sets of both languages. On this overlap set, the paper defines a cosine-style similarity:
\[
\text{sim}(\mathcal{T}_u,\mathcal{T}_v) = \frac{\boldsymbol{f_u} \cdot \boldsymbol{f_v}}{\lVert \boldsymbol{f_u} \rVert \lVert \boldsymbol{f_v} \rVert},
\]
where \(\boldsymbol{f_u}\) and \(\boldsymbol{f_v}\) are activation-frequency vectors restricted to \(\mathcal{T}_{u,v}\).

This construction underlies what the paper calls the model’s **internal linguistic spectrum**. The proposed interpretation is mechanistic: overlap neurons encode shared processing pathways between languages and may capture shared semantic information even across families. The paper further associates **middle layers** with semantic understanding and **final layers** with generation or language coding for next-token prediction. Its logit-lens analysis suggests that during translation, latent states in middle layers move toward representations associated with a high-resource language such as English. This supports the claim that a useful bridge language is not simply a genealogically related language, but a language whose internal neuron behavior matches the source–target overlap structure in the model [2508.17078].

The resulting linguistic spectrum is only partly human-like. Heatmaps partly align with language genealogy within families, but not uniformly across families. The paper highlights the example that **Arabic can be slightly more similar to French than to Hebrew in LLaMA 3’s overlap-neuron space**, which it attributes plausibly to training-distribution effects rather than to linguistic genealogy.

## 3. HSIC-based bridge selection and the BridgeX-ICL pipeline

BridgeX-ICL uses probe data derived from **ground-truth bilingual lexicons** to ensure what it calls **“full activation of these anchored neurons.”** The method uses **MUSE bilingual dictionaries** for bridge-selection probing and **FLORES+** for language-neuron identification. For each language pair \((L_1,L_2)\), it collects \(d=100\) word pairs that the LLM can translate accurately, prompts the model in both directions, and records FFN activations during generation rather than passive token reading. The stated purpose is to make the neuron signal reflect validated bilingual competence rather than noise [2508.17078].

Given source \(L_s\) and target \(L_t\), let \(\mathcal{T}_{s,t}\) be their overlap neurons. Using the bilingual probes, the method constructs an activation matrix
\[
\mathbf{X} \in \mathbb{R}^{|\mathcal{T}_{s,t}|\times 2d}
\]
for overlap neurons. For a candidate bridge language \(L_y\), it constructs
\[
\mathbf{Y} \in \mathbb{R}^{|\bar{\mathcal{T}_{y}}|\times 2d},
\]
where the paper states that \(\bar{\mathcal{T}_{y}}\) contains neurons specific to bridge language \(L_y\), excluding neurons already in the source–target overlap set and excluding neurons shared with another bridge language \(L_{y'}\). After average pooling so that \(\mathbf{X}\) and \(\mathbf{Y}\) have the same row dimension \(n\), dependence is measured with **HSIC**:
\[
\text{HSIC}(\mathbf{X},\mathbf{Y}) = n^{-2}\text{Tr}(\mathbf{KHLH}),
\]
where \(\mathbf{H}=\mathbf{I}_{n\times n}- \frac{1}{n} \mathbf{1}_n \mathbf{1}_n^\top\).

Rather than using only whole-matrix HSIC, the paper adopts a **bidirectional maximum matching strategy**:
\[
\begin{aligned}
& \mathrm{HSIC}(\bar{\mathcal{T}_{y}},\mathcal{T}_{s,t})= \\
& \frac{1}{2}\left(\max_i \mathrm{HSIC}(\mathbf{x}_i,\mathbf{Y})+\max_j\mathrm{HSIC}(\mathbf{X},\mathbf{y}_j)\right),
\end{aligned}
\]
and then averages over selected layers:
\[
\begin{aligned}
p(L_y|L_s\text{-}L_t)=\frac{1}{K}\sum_{i=1}^K \text{HSIC}\left( \bar{\mathcal{T}_{y}}(i),\mathcal{T}_{s,t}(i) \right).
\end{aligned}
\]
The chosen bridge is
\[
L^* = \arg\max_{L_y \in \mathcal{L}_\text{candidate}} p(L_y|L_s\text{-}L_t).
\]

Layer selection is itself data-driven. Using embedding semantic similarity across layers, the method selects stable middle-layer regions: **layers 10–21 for LLaMA 3** and **layers 15–23 for Mistral**. End to end, the pipeline is: prepare bilingual resources; build probe data; identify language neuron sets \(\mathcal{T}_k\); construct source–target overlap neurons \(\mathcal{T}_{s,t}\); compute bridge-specific neuron sets; evaluate candidate bridges with the HSIC score; choose \(L^*\); and finally prompt the downstream task in a **zero-shot bridge format** using the selected bridge language [2508.17078].

## 4. Experimental design and evaluation setting

The empirical study covers **two cross-lingual tasks**: **Bilingual Lexicon Induction (BLI)** and **Machine Reading Comprehension (MRC)** using **Belebele**. The evaluated models are **LLaMA-3-8B** and **Mistral-7B-Instruct-v0.3**. The language inventory contains **15 languages from 7 families**: Indo-European (**En, De, Fr, It, Pt, Es**), Uralic (**Fi, Hu**), Afro-Asiatic (**Ar, He**), Austronesian (**Id, Tl**), Sino-Tibetan (**Zh**), Japonic (**Ja**), and Niger-Congo (**Sw**). The designated low-resource targets are **Hebrew, Tagalog, Swahili, and Japanese** [2508.17078].

The paper evaluates **15 source–target pairs**: **Zh-Ja, Zh-He, Zh-Tl, Zh-Sw, Ar-Ja, Ar-He, Ar-Tl, Ar-Sw, Id-Ja, Id-He, Id-Tl, Id-Sw, En-He, En-Tl, En-Sw**. These include both **high-low** and **moderate-low** settings. For dictionaries, the method uses **MUSE** where available and constructs missing pairs through **English pivoting**; for **Swahili**, it uses **wiktionary_bli** to build **En-Sw**. Word pairs are validated with both Google and Microsoft translators, and **1,000 consistent pairs** are kept per language pair.

Bridge candidates are restricted to six well-supported Indo-European languages: **En, De, Fr, It, Pt, Es**. Metrics are **Precision@1** for BLI and **multiple-choice accuracy** for MRC. Baselines comprise **Zero-shot**, **Few-shot** (1, 2, 3, 4 shots), and **Zero-shot with bridge**. Bridge-selection baselines include **Human source**, **Human target**, **English bridge**, **Sharing matter**, **IoU score**, and **LAPE\(_{\text{overlap}}\)**. The IoU baseline is defined as
\[
\mathrm{IoU}(\mathcal{T}_{u},\mathcal{T}_{v}) = (\mathcal{T}_{u}\cap\mathcal{T}_{v}) / (\mathcal{T}_{u}\cup\mathcal{T}_{v}).
\]
This design isolates the contribution of neuron-guided bridge selection rather than few-shot example accumulation [2508.17078].

## 5. Quantitative findings

The paper reports average improvements of BridgeX-ICL over zero-shot baselines of **+6.02%** and **+5.25%** across the two tasks; in the task-wise discussion it reports **+6.03%** on MRC for **LLaMA 3** and **+4.48%** for **Mistral**. On BLI, BridgeX-ICL often reaches performance comparable to **two-shot X-ICL** while remaining zero-shot [2508.17078].

The strongest gains appear on several BLI pairs. For **LLaMA 3**, BridgeX-ICL improves over zero-shot by **+10.80** on **Zh-Ja**, **+12.60** on **Zh-He**, **+11.40** on **Zh-Tl**, **+10.70** on **Ar-Ja**, **+17.50** on **Ar-He**, **+16.60** on **Id-He**, and **+14.90** on **En-He**. For **Mistral**, reported improvements include **+7.90** on **Zh-Ja**, **+8.60** on **Zh-He**, **+6.40** on **Zh-Tl**, **+8.90** on **Ar-Ja**, and **+8.60** on **Ar-Tl**. The method is not uniformly positive: the paper also reports negative cases, including **En-Tl: -1.30** and **En-Sw: -2.60** for LLaMA 3, and **Id-He: -3.30** and **En-He: -0.10** for Mistral.

Few-shot prompting is described as unstable, especially for Mistral: **one-shot can be worse than zero-shot**, gains tend to saturate by **3 shots**, and additional examples beyond 3 often do not help. This is one of the paper’s practical arguments for bridge prompting as a cheaper intervention than few-shot demonstration scaling.

Bridge selection itself is model-specific. **English** is selected as the optimal bridge in **9/15** pairs, but not universally; **Es, Fr, Pt, and De** are also selected, and the preferred bridge differs across LLaMA 3 and Mistral. Human-genealogy-based bridge choices perform worst overall. The appendix comparison between **HSIC** and cosine similarity further supports the proposed metric: for example, on **Zh-Ja** with LLaMA 3, HSIC selects **En** and yields **77.90**, whereas cosine selects **De** and yields **76.60**; on **Id-Tl**, HSIC selects **Fr** and yields **61.00**, whereas cosine selects **De** and yields **57.00**.

A key ablation concerns probe construction. Replacing the MUSE-based activated bilingual probes with simpler FLORES+-derived bilingual tokens generally reduces performance. Reported examples include **Ar-Tl: 52.00 \(\rightarrow\) 56.80** and **En-He: 66.60 \(\rightarrow\) 71.80** for LLaMA 3, and **Id-Ja: 50.80 \(\rightarrow\) 58.00** and **Id-He: 33.40 \(\rightarrow\) 38.40** for Mistral. This is the main empirical support for the claim that validated bilingual generation better identifies useful overlap neurons [2508.17078].

## 6. Interpretation, limitations, and place within ICL research

BridgeX-ICL advances a specific view of multilingual transfer: low-resource transfer is mediated not only by language-specific subnetworks but also by **shared multilingual circuitry**. The method operationalizes this by turning overlap-neuron geometry into a bridge-language selector, thereby making bridge prompting depend on the model’s **internal linguistic spectrum** rather than on external language-family heuristics. A plausible implication is that cross-lingual prompting should be treated as a model-internal alignment problem as much as a linguistic one [2508.17078].

The paper also states clear limitations. It evaluates only **15 language pairs** and **4 low-resource targets**; the inferred linguistic distance is **probe-induced and qualitative, not a precise universal metric**; the method depends on **high-quality probe data** and on the target language being present enough in the LLM to yield meaningful activations; and candidate bridges are limited to **well-supported Indo-European languages**. It further notes a substantive failure mode: bridge prompting can hurt when the selected bridge does not match the model’s actual transfer path or when the source–target pair is already well handled.

Within the broader ICL literature, BridgeX-ICL belongs to a family of methods that modify inference by exploiting internal or structural information rather than by standard fine-tuning. Adjacent work has aligned supervised fine-tuning with ICL through internal activation matching in **IA2** [2509.22621], reduced prompt-template arbitrariness through reusable soft-token schemas in **ICL Markup** [2312.07405], probed inference-time learning via reversible and non-reversible substitution ciphers in **ICL CIPHERS** [2504.19395], and improved fine-grained emotion recognition by replacing semantic retrieval with task-specific prototype selection and exclusionary candidate pruning in **E-ICL** [2406.02642]. BridgeX-ICL is distinct within this landscape because it does not alter parameters or prompt structure directly; instead, it uses **overlap neurons** and **HSIC-based bridge selection** to decide how a multilingual prompt should be framed.

In that sense, BridgeX-ICL is both a practical zero-shot prompting method and a mechanistic claim about multilingual LLMs. Its central assertion is that the best bridge language is the one that best matches the model’s own shared neuron structure for a source–target pair, not necessarily the one closest under human linguistic taxonomy.

Source: https://www.emergentmind.com/topics/bridgex-icl