MUREL: A Multicultural Resource for LLMs

• MUREL is a curated corpus that standardizes culture-specific data from six languages, enabling precise localization of cultural representations in LLMs.
• It aggregates 85.2 million tokens from 69 datasets organized into ideational, linguistic, and social elements to support culture-conditioned analysis.
• Experimental findings reveal that ablation of culture-specific neurons significantly increases model perplexity, emphasizing their functional role.

Searching arXiv for the relevant papers and related context. MUREL is a curated corpus introduced to study cultural representations in multilingual LLMs. The name stands for MUlticultural Resource for Evaluating LLMs, and its central function is to provide culture-salient, standardized inputs for localizing and evaluating culture-specific neurons and pure culture-specific neurons in transformer feed-forward networks. In the underlying methodology, Wikipedia is used to identify language-specific neurons, whereas MUREL is used as the culture-side counterpart for estimating culture-conditioned activation statistics, computing Culture Activation Probability Entropy (CAPE), and evaluating inference-time neuron interventions on held-out cultural text (Namazifard et al., 4 Aug 2025).

1. Nomenclature and scope

The term MUREL denotes the corpus resource introduced in "Isolating Culture Neurons in Multilingual LLMs" (Namazifard et al., 4 Aug 2025). It is distinct from MuRel, the Multimodal Relational network proposed for visual question answering, which replaces standard scalar attention maps with a vectorial, relational reasoning mechanism over object regions (Cadene et al., 2019). It is also distinct from MURE, the MCNP Utility for Reactor Evolution, a general-purpose reactor evolution code built on top of the MCNP Monte Carlo neutron-transport engine for time-dependent isotopic evolution and fission-rate extraction in reactor antineutrino studies (Jones et al., 2011).

Within the culture-neuron literature, MUREL is not a model architecture but a dataset collection. Its role is infrastructural: it supplies the balanced, culture-conditioned corpora required to compute activation probabilities over cultures, disentangle culture from language at the neuron level, and test the functional consequences of ablating identified neuron subsets (Namazifard et al., 4 Aug 2025).

2. Corpus composition and coverage

MUREL consists of 69 datasets spanning 6 cultural groups, with a total size of 85.2 million tokens, measured as Gemma-3 tokens. The average size is 14.2M tokens per culture. The six cultures are operationalized via six languages—English, German, Danish, Chinese, Russian, Persian—selected to span geography, typology, and resource level. In this operationalization, “culture” is effectively approximated by the national or linguistic culture associated with each language (Namazifard et al., 4 Aug 2025).

Culture / language	Tokens (k)
EN (English)	14,262
DE (German)	19,891
DA (Danish)	11,383
ZH (Chinese)	16,511
RU (Russian)	12,405
FA (Persian)	10,769

The selection criteria are explicitly culture-aware. For each of the six cultures, the corpus aggregates datasets whose creation, annotation, or content is specific to that culture. The sources are public and licensed; the resource is described as using public, licensed sources only, and the constituent datasets are used in accordance with their respective open licenses for research purposes only. For neuron localization, the experiments use a balanced 10M tokens per culture. For evaluation and ablation, they retain a held-out test set of 100k tokens per culture. This balancing is intended to reduce confounding by corpus size when comparing activation statistics across cultures (Namazifard et al., 4 Aug 2025).

3. Taxonomic structure and data preparation

MUREL is organized as a structured collection rather than a monolithic dataset. Its internal organization follows a cultural taxonomy with three top-level categories: Ideational Elements, Linguistic Elements, and Social Elements. This structure is used to ensure that culture is not reduced to a single domain such as politics, food, or values alone (Namazifard et al., 4 Aug 2025).

Ideational Elements cover high-level cultural concepts and knowledge. The stated subcategories are Concepts, Knowledge, Values, Norms and Morals, and Artifacts. Representative source types include metaphor, proverb, idiom, and irony datasets; cultural QA and knowledge resources; survey-based value sources such as PEW, WVS, PCT, and Hofstede; norm and moral benchmarks; and literary or culturally grounded narrative sources.

Linguistic Elements capture cultural variation expressed primarily through language. The listed subcategories are Dialects and Styles, Registers, Genres, including dialect identification resources and datasets on politeness, formality, slang, genre, and style comparison.

Social Elements represent culture as realized in social interaction and demographics. The subcategories are Relationship, Context, Communicative Goals, and Demographics, with source types including relational dialogue corpora, contextual framing resources, norm-adherence or norm-violation datasets, and corpora tracking demographic variation.

The preparation pipeline emphasizes textual compatibility with LLM evaluation. Some structured resources, especially surveys and questionnaires, are converted into full textual statements so that each line becomes a culturally meaningful sentence suitable for perplexity and activation measurement. The paper gives the example of a masked survey item such as “Familie ist [MASK] in meinem Leben” being rewritten into complete assertions such as “Familie ist wichtig in meinem Leben.” This conversion is intended to make each response a stand-alone cultural assertion rather than a masked-language-model fragment (Namazifard et al., 4 Aug 2025).

4. Role in culture-neuron localization

MUREL is the dataset that enables the extension from Language Activation Probability Entropy (LAPE) to Culture Activation Probability Entropy (CAPE). In the reported framework, a neuron is defined in a transformer FFN/MLP block as one column of $\mathbf{W}^{(\ell)}_1$ followed by the activation function. For a GLU-style feed-forward block, the layer computation is written as

$$\mathbf{h}^{\ell} = \left( \operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right) \otimes \tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_3 \right) \mathbf{W}^{(\ell)}_2.$$

Language-specific neurons are first estimated from balanced Wikipedia corpora. For neuron $j$ in layer $\ell$ and language $k$, the activation probability is

$$p^k_{\ell,j} = \mathbb{E}\!\left[ \mathbb{I}\!\left( \operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right)_j > 0 \right) \,\middle|\, \text{language } k \right].$$

After normalization across languages, LAPE is the entropy of that distribution. Low-entropy neurons are language-specific, and for each language the set $\mathbb{L}_k$ is formed by taking the bottom 1% of neurons by LAPE, subject to the additional condition that at least one language has sufficiently high activation probability.

MUREL enters at the next step. For culture $m$, neuron $j$, and layer $\ell$, the culture-conditioned activation probability is

$$\mathbf{h}^{\ell} = \left( \operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right) \otimes \tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_3 \right) \mathbf{W}^{(\ell)}_2.$$0

These expectations are estimated over MUREL’s balanced cultural corpora. After normalization across cultures, CAPE is defined as the entropy of the resulting distribution. Culture-specific neurons for culture $$\mathbf{h}^{\ell} = \left( \operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right) \otimes \tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_3 \right) \mathbf{W}^{(\ell)}_2.$$1 are then selected as

$$\mathbf{h}^{\ell} = \left( \operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right) \otimes \tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_3 \right) \mathbf{W}^{(\ell)}_2.$$2

with the implementation mirroring the LAPE procedure by taking the bottom 1% of neurons by CAPE. The key disentangling step is set-theoretic. For a culture $$\mathbf{h}^{\ell} = \left( \operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right) \otimes \tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_3 \right) \mathbf{W}^{(\ell)}_2.$$3 associated with language $$\mathbf{h}^{\ell} = \left( \operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right) \otimes \tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_3 \right) \mathbf{W}^{(\ell)}_2.$$4,

$$\mathbf{h}^{\ell} = \left( \operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right) \otimes \tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_3 \right) \mathbf{W}^{(\ell)}_2.$$5

defines pure culture-specific neurons, while

$$\mathbf{h}^{\ell} = \left( \operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right) \otimes \tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_3 \right) \mathbf{W}^{(\ell)}_2.$$6

defines compound language-and-culture neurons. The paper is explicit that there is no separate explicit label that directly disentangles culture from language at the datapoint level; the disentanglement is achieved at the neuron level through these set operations (Namazifard et al., 4 Aug 2025).

5. Experimental use and principal findings

The reported experiments study four transformer LLMs at inference time only, without fine-tuning on MUREL: Llama-2-7B, Llama-3.1-8B, Qwen2.5-7B, and Gemma-3-12B. MUREL is used in two phases. In the localization phase, 10M MUREL tokens per culture are passed through the model to compute $$\mathbf{h}^{\ell} = \left( \operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right) \otimes \tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_3 \right) \mathbf{W}^{(\ell)}_2.$$7, CAPE, and the culture-neuron sets. In the evaluation phase, a held-out 100k tokens per culture are used to measure perplexity under neuron ablations. The intervention itself consists of zeroing neuron activations for selected subsets, including $$\mathbf{h}^{\ell} = \left( \operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right) \otimes \tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_3 \right) \mathbf{W}^{(\ell)}_2.$$8, $$\mathbf{h}^{\ell} = \left( \operatorname{act\_fn}\!\left(\tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_1\right) \otimes \tilde{\mathbf{h}}^{\ell}\mathbf{W}^{(\ell)}_3 \right) \mathbf{W}^{(\ell)}_2.$$9, $j$0, the compound intersection, or same-size random controls, and then measuring the change in perplexity relative to baseline (Namazifard et al., 4 Aug 2025).

Several empirical results are attached directly to this setup. Across the models, the authors identify 3,523 specialized neurons in Llama-2-7B, 4,588 in Llama-3.1-8B, 1,004 in Qwen2.5-7B, and 7,373 in Gemma-3-12B; these totals include both language and culture neurons. Different cultures map to largely distinct neuron subsets, with only limited overlap through the compound sets. Lower-resource languages such as Persian and Danish tend to have more language- and culture-specific neurons. On average, 56.7% of culture-specific neurons are pure culture-specific, and the paper reports that no “generic culture neurons” are shared across all cultures, formally

$j$1

Layer-wise analyses show that in the multilingual models—Llama-3.1-8B, Qwen2.5-7B, and Gemma-3-12B—both language-specific and culture-specific neurons are concentrated primarily in the upper layers. Pure culture-specific neurons follow the same pattern but are sparser. In Llama-2-7B, by contrast, the distribution is described as bimodal, with peaks in early and top layers.

The ablation results are used to establish functional specificity. Ablating all culture-specific neurons for a culture produces the largest increase in perplexity on that culture’s texts. Ablating pure culture neurons also substantially increases perplexity; in Llama-3.1-8B, pure culture neurons alone account for about 76.3% of the total effect of ablating all culture neurons. The reported off-diagonal effects in the $j$2 heatmaps are near zero, indicating limited cross-cultural interference, and same-size random ablations have negligible effect. This supports the conclusion that culture neurons are both localized and functionally specific within the tested models (Namazifard et al., 4 Aug 2025).

6. Interpretive significance, applications, and limitations

MUREL is presented as an enabling resource for culture-aware mechanistic interpretability. Because it provides large, balanced, culturally organized corpora, it allows direct measurement of internal cultural biases and propensities rather than relying exclusively on external behavior. The paper states that localizing neurons associated with particular cultures’ norms, values, idioms, and related phenomena can support interventions such as selectively dampening or re-balancing activations tied to harmful or over-represented cultural patterns, adjusting cultural propensities, and checking whether the density and functional impact of culture neurons are comparable across cultures. The authors frame these possibilities as promoting fairness, inclusivity, and alignment (Namazifard et al., 4 Aug 2025).

The limitations are also explicit. MUREL covers only six languages/cultures, leaving many regions and language families absent. It is assembled from existing public datasets, which may themselves be biased toward particular demographics or online populations. Culture is approximated through language and dataset origin, so intra-cultural diversity is not systematically modeled. The dynamic, time-varying character of culture is not explicitly normalized. Methodologically, the neuron-localization procedure depends on entropy metrics, binary activation $j$3, and specific token-sampling choices such as 10M tokens per culture, which may miss more distributed or context-dependent representations. Evaluation is also narrow: the paper mainly measures perplexity on MUREL, rather than downstream QA, dialogue, or human-facing generation quality.

Within those limits, MUREL functions as a research substrate for probing, comparing, and intervening on cultural representations in multilingual LLMs. The associated work reports that code and data are available at https://github.com/namazifard/Culture_Neurons, and describes practical uses including rerunning CAPE on other multilingual models, benchmarking cultural robustness, and combining neuron localization with model-editing methods for culture-sensitive steering (Namazifard et al., 4 Aug 2025).