---
title: 'MUREL: A Multicultural Resource for LLMs'
url: https://www.emergentmind.com/topics/murel
type: topic
---

# MUREL: A Multicultural Resource for LLMs

Searching arXiv for the relevant papers and related context.
MUREL is a curated corpus introduced to study cultural representations in multilingual large language models. The name stands for **MUlticultural Resource for Evaluating Language Models**, and its central function is to provide culture-salient, standardized inputs for localizing and evaluating **culture-specific neurons** and **pure culture-specific neurons** in transformer feed-forward networks. In the underlying methodology, Wikipedia is used to identify language-specific neurons, whereas MUREL is used as the culture-side counterpart for estimating culture-conditioned activation statistics, computing **Culture Activation Probability Entropy (CAPE)**, and evaluating inference-time neuron interventions on held-out cultural text [2508.02241].

## 1. Nomenclature and scope

The term **MUREL** denotes the corpus resource introduced in "Isolating Culture Neurons in Multilingual Large Language Models" [2508.02241]. It is distinct from **MuRel**, the **Multimodal Relational** network proposed for visual question answering, which replaces standard scalar attention maps with a vectorial, relational reasoning mechanism over object regions [1902.09487]. It is also distinct from **MURE**, the **MCNP Utility for Reactor Evolution**, a general-purpose reactor evolution code built on top of the MCNP Monte Carlo neutron-transport engine for time-dependent isotopic evolution and fission-rate extraction in reactor antineutrino studies [1109.5379].

Within the culture-neuron literature, MUREL is not a model architecture but a **dataset collection**. Its role is infrastructural: it supplies the balanced, culture-conditioned corpora required to compute activation probabilities over cultures, disentangle culture from language at the neuron level, and test the functional consequences of ablating identified neuron subsets [2508.02241].

## 2. Corpus composition and coverage

MUREL consists of **69 datasets** spanning **6 cultural groups**, with a total size of **85.2 million tokens**, measured as **Gemma-3 tokens**. The average size is **14.2M tokens per culture**. The six cultures are operationalized via six languages—**English, German, Danish, Chinese, Russian, Persian**—selected to span **geography**, **typology**, and **resource level**. In this operationalization, “culture” is effectively approximated by the national or linguistic culture associated with each language [2508.02241].

| Culture / language | Tokens (k) |
|---|---:|
| EN (English) | 14,262 |
| DE (German) | 19,891 |
| DA (Danish) | 11,383 |
| ZH (Chinese) | 16,511 |
| RU (Russian) | 12,405 |
| FA (Persian) | 10,769 |

The selection criteria are explicitly culture-aware. For each of the six cultures, the corpus aggregates datasets whose **creation**, **annotation**, or **content** is specific to that culture. The sources are public and licensed; the resource is described as using **public, licensed sources only**, and the constituent datasets are used in accordance with their respective open licenses for research purposes only. For neuron localization, the experiments use a **balanced 10M tokens per culture**. For evaluation and ablation, they retain a **held-out test set of 100k tokens per culture**. This balancing is intended to reduce confounding by corpus size when comparing activation statistics across cultures [2508.02241].

## 3. Taxonomic structure and data preparation

MUREL is organized as a structured collection rather than a monolithic dataset. Its internal organization follows a cultural taxonomy with three top-level categories: **Ideational Elements**, **Linguistic Elements**, and **Social Elements**. This structure is used to ensure that culture is not reduced to a single domain such as politics, food, or values alone [2508.02241].

**Ideational Elements** cover high-level cultural concepts and knowledge. The stated subcategories are **Concepts**, **Knowledge**, **Values**, **Norms and Morals**, and **Artifacts**. Representative source types include metaphor, proverb, idiom, and irony datasets; cultural QA and knowledge resources; survey-based value sources such as PEW, WVS, PCT, and Hofstede; norm and moral benchmarks; and literary or culturally grounded narrative sources.

**Linguistic Elements** capture cultural variation expressed primarily through language. The listed subcategories are **Dialects** and **Styles, Registers, Genres**, including dialect identification resources and datasets on politeness, formality, slang, genre, and style comparison.

**Social Elements** represent culture as realized in social interaction and demographics. The subcategories are **Relationship**, **Context**, **Communicative Goals**, and **Demographics**, with source types including relational dialogue corpora, contextual framing resources, norm-adherence or norm-violation datasets, and corpora tracking demographic variation.

The preparation pipeline emphasizes textual compatibility with LLM evaluation. Some structured resources, especially surveys and questionnaires, are converted into **full textual statements** so that each line becomes a culturally meaningful sentence suitable for perplexity and activation measurement. The paper gives the example of a masked survey item such as *“Familie ist [MASK] in meinem Leben”* being rewritten into complete assertions such as *“Familie ist wichtig in meinem Leben.”* This conversion is intended to make each response a stand-alone cultural assertion rather than a masked-language-model fragment [2508.02241].

## 4. Role in culture-neuron localization

MUREL is the dataset that enables the extension from **Language Activation Probability Entropy (LAPE)** to **Culture Activation Probability Entropy (CAPE)**. In the reported framework, a neuron is defined in a transformer FFN/MLP block as one column of $\mathbf{W}^{(\ell)}_1$ followed by the activation function. For a GLU-style feed-forward block, the layer computation is written as

$$
\mathbf{h}^{\ell}
=
\left(
\operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right)
\otimes
\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_3
\right)
\mathbf{W}^{(\ell)}_2.
$$

Language-specific neurons are first estimated from balanced Wikipedia corpora. For neuron $j$ in layer $\ell$ and language $k$, the activation probability is

$$
p^k_{\ell,j}
=
\mathbb{E}\!\left[
\mathbb{I}\!\left(
\operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right)_j > 0
\right)
\,\middle|\,
\text{language } k
\right].
$$

After normalization across languages, **LAPE** is the entropy of that distribution. Low-entropy neurons are language-specific, and for each language the set $\mathbb{L}_k$ is formed by taking the **bottom 1% of neurons by LAPE**, subject to the additional condition that at least one language has sufficiently high activation probability.

MUREL enters at the next step. For culture $m$, neuron $j$, and layer $\ell$, the culture-conditioned activation probability is

$$
q^m_{\ell,j}
=
\mathbb{E}\!\left[
\mathbb{I}\!\left(
\operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right)_j > 0
\right)
\,\middle|\,
\text{culture } m
\right].
$$

These expectations are estimated over MUREL’s balanced cultural corpora. After normalization across cultures, **CAPE** is defined as the entropy of the resulting distribution. Culture-specific neurons for culture $m$ are then selected as

$$
\mathbb{C}_m
=
\{ v \in \mathbb{N} \mid \operatorname{CAPE}(v) \le \tau_{\text{cult}} \},
$$

with the implementation mirroring the LAPE procedure by taking the **bottom 1%** of neurons by CAPE. The key disentangling step is set-theoretic. For a culture $m$ associated with language $k$,

$$
\mathbb{P}_m
=
\mathbb{C}_m \setminus \mathbb{L}_k
$$

defines **pure culture-specific neurons**, while

$$
\mathbb{L}_k \cap \mathbb{C}_m
$$

defines **compound language-and-culture neurons**. The paper is explicit that there is **no separate explicit label that directly disentangles culture from language at the datapoint level**; the disentanglement is achieved at the **neuron level** through these set operations [2508.02241].

## 5. Experimental use and principal findings

The reported experiments study four transformer LLMs at **inference time only**, without fine-tuning on MUREL: **Llama-2-7B**, **Llama-3.1-8B**, **Qwen2.5-7B**, and **Gemma-3-12B**. MUREL is used in two phases. In the **localization phase**, **10M MUREL tokens per culture** are passed through the model to compute $q^m_{\ell,j}$, CAPE, and the culture-neuron sets. In the **evaluation phase**, a **held-out 100k tokens per culture** are used to measure perplexity under neuron ablations. The intervention itself consists of **zeroing neuron activations** for selected subsets, including $\mathbb{L}_k$, $\mathbb{C}_m$, $\mathbb{P}_m$, the compound intersection, or same-size random controls, and then measuring the **change in perplexity** relative to baseline [2508.02241].

Several empirical results are attached directly to this setup. Across the models, the authors identify **3,523 specialized neurons in Llama-2-7B**, **4,588 in Llama-3.1-8B**, **1,004 in Qwen2.5-7B**, and **7,373 in Gemma-3-12B**; these totals include both language and culture neurons. Different cultures map to **largely distinct neuron subsets**, with only limited overlap through the compound sets. Lower-resource languages such as **Persian and Danish** tend to have **more language- and culture-specific neurons**. On average, **56.7% of culture-specific neurons are pure culture-specific**, and the paper reports that no “generic culture neurons” are shared across all cultures, formally

$$
\bigcap_m \mathbb{P}_m = \emptyset.
$$

Layer-wise analyses show that in the multilingual models—**Llama-3.1-8B, Qwen2.5-7B, and Gemma-3-12B**—both language-specific and culture-specific neurons are concentrated primarily in the **upper layers**. **Pure culture-specific neurons** follow the same pattern but are sparser. In **Llama-2-7B**, by contrast, the distribution is described as **bimodal**, with peaks in early and top layers.

The ablation results are used to establish functional specificity. **Ablating all culture-specific neurons for a culture** produces the **largest increase in perplexity** on that culture’s texts. **Ablating pure culture neurons** also substantially increases perplexity; in **Llama-3.1-8B**, pure culture neurons alone account for **about 76.3%** of the total effect of ablating all culture neurons. The reported **off-diagonal** effects in the $\Delta\text{PPL}$ heatmaps are near zero, indicating limited cross-cultural interference, and same-size **random ablations** have negligible effect. This supports the conclusion that culture neurons are both localized and functionally specific within the tested models [2508.02241].

## 6. Interpretive significance, applications, and limitations

MUREL is presented as an enabling resource for **culture-aware mechanistic interpretability**. Because it provides large, balanced, culturally organized corpora, it allows direct measurement of internal cultural biases and propensities rather than relying exclusively on external behavior. The paper states that localizing neurons associated with particular cultures’ norms, values, idioms, and related phenomena can support interventions such as selectively dampening or re-balancing activations tied to harmful or over-represented cultural patterns, adjusting cultural propensities, and checking whether the density and functional impact of culture neurons are comparable across cultures. The authors frame these possibilities as promoting **fairness, inclusivity, and alignment** [2508.02241].

The limitations are also explicit. MUREL covers only **six languages/cultures**, leaving many regions and language families absent. It is assembled from existing public datasets, which may themselves be biased toward particular demographics or online populations. Culture is approximated through **language and dataset origin**, so intra-cultural diversity is not systematically modeled. The dynamic, time-varying character of culture is not explicitly normalized. Methodologically, the neuron-localization procedure depends on **entropy metrics**, **binary activation $>0$**, and specific token-sampling choices such as **10M tokens per culture**, which may miss more distributed or context-dependent representations. Evaluation is also narrow: the paper mainly measures **perplexity on MUREL**, rather than downstream QA, dialogue, or human-facing generation quality.

Within those limits, MUREL functions as a research substrate for probing, comparing, and intervening on cultural representations in multilingual LLMs. The associated work reports that **code and data** are available at **https://github.com/namazifard/Culture_Neurons**, and describes practical uses including rerunning CAPE on other multilingual models, benchmarking cultural robustness, and combining neuron localization with model-editing methods for culture-sensitive steering [2508.02241].

Source: https://www.emergentmind.com/topics/murel