CultureMarkers: Culturally Tagged LLM Data

• CultureMarkers is a culturally tagged dataset and framework that explicitly annotates cultural signals across various LLM training stages.
• It uses a multidimensional tagging scheme across five dimensions to distinguish culture from language, geolocation, and domain-specific content.
• Empirical results show that explicit marker augmentation boosts cultural alignment, while cultural signals sharply decline from pretraining to reasoning stages.

CultureMarkers is a culturally tagged dataset and tagging framework for LLM training data. It was introduced to make culture explicit across the model pipeline, on the premise that if culture is not visible in training data, it cannot be reliably learned or preserved later. The resource contains 5.6M tagged samples across 10 datasets spanning pretraining, fine-tuning, alignment, reasoning, and real-world chat use, and it is paired with an analysis of how cultural signals change across those stages. Its central thesis is the existence of a “culture funnel”: pretraining contains the broadest cultural grounding, whereas post-training stages increasingly prioritize reasoning, coding, math, and alignment objectives, so explicit cultural signals decline sharply during SFT $\rightarrow$ alignment $\rightarrow$ reasoning (Sahu et al., 11 Jun 2026).

1. Conceptual basis and research problem

CultureMarkers was designed to address a measurement problem in cultural NLP. The paper argues that culture is usually implicit in training data documentation, that existing datasets and data cards rarely quantify it, that cultural content is often conflated with language coverage, and that LLM training data is typically optimized for general utility in ways that can suppress culturally situated examples. In response, CultureMarkers makes culture a first-class annotation target so that one can compare cultural content across pipeline stages, study its co-occurrence with domain, task intent, language, and geolocation, and test whether explicit cultural markers improve downstream cultural performance (Sahu et al., 11 Jun 2026).

The framework is organized around an upstream claim rather than a benchmark-only claim. The paper maintains that current cultural alignment approaches focus too heavily on inference-time interventions and implicitly assume that models already contain sufficient cultural knowledge. CultureMarkers rejects that assumption. It treats cultural alignment as a data-pipeline problem: the model’s cultural behavior depends on what survives the transition from pretraining to post-training, not only on prompting or alignment-time wrappers.

A common misconception addressed by the paper is that multilinguality alone solves cultural representation. The reported result is narrower: multilinguality helps with geographic diversity of cultural knowledge, but it does not guarantee balanced cultural representation. Another misconception is that language coverage can stand in for culture coverage. CultureMarkers is explicitly constructed to separate those notions by tagging language, geolocation, and cultural content as distinct dimensions.

2. Tagging framework and annotation scheme

CultureMarkers uses a multidimensional tagging framework with five dimensions: Cultural dimension, Domain, Task intent, Geolocation, and Language. Task intent is used only for post-training datasets. For pretraining corpora, the authors tag the entire text. For post-training data, they tag only the input prompt/instruction, not the output/completion. For conversational datasets, they annotate only the user side. The paper therefore measures opportunities for cultural learning, not the cultural adequacy of model responses (Sahu et al., 11 Jun 2026).

The cultural taxonomy includes CultureAsKnowledge, CultureAsPreference, CultureAsDynamics, CultureAsBias, GeneralCulture, NoCulture, and Unspecified. Their meanings are defined as follows. CultureAsKnowledge covers factual or symbolic cultural content such as holidays, foods, customs, named entities, and local traditions. CultureAsPreference covers shared values, norms, and political or social attitudes. CultureAsDynamics treats culture as something enacted in context, interaction, or communication. CultureAsBias covers stereotypes, discrimination, or harmful generalizations. GeneralCulture is used for culturally grounded content that does not cleanly fit the other categories. NoCulture denotes no cultural component, and Unspecified is reserved for uncertain cases. The paper emphasizes that these categories are not mutually exclusive in theory, but each example is assigned a single best tag in practice.

The Domain taxonomy assigns one domain per sample, using classes such as HumanitiesArts, Sciences, Technology, SocialSciences, Medical, Finance, Legal, Conversation, Code, Math, and Unspecified. The Task intent taxonomy includes WritingCommunication, CreativeWriting, AcademicWriting, CodingTechnicalHelp, Translation, Summarization, ExplanationLearning, InformationExtraction, EditingRewriting, Classification, ReasoningProblemSolving, PracticalGuidance, LegalAdministrative, MedicalHealth, JobCareer, BusinessFinance, LocalInformation, LanguageLearning, Conversation, and Unspecified.

Automatic annotation uses Command-A for tagging and FastText LangID for language identification, with validation against human annotations. Agreement is reported using Krippendorff’s $\alpha$. The paper states that Geolocation had the strongest agreement, while Culture and Domain were more variable, and that agreement differed by language. It further interprets disagreement as reflecting real cultural ambiguity, not merely annotation failure.

The four-way cultural framing in CultureMarkers closely matches the anthropological taxonomy of culture-as-knowledge, culture-as-preference, culture-as-dynamics, and culture-as-bias used to critique existing culture benchmarks (AlKhamissi et al., 7 Oct 2025). In CultureMarkers, however, that taxonomy is used not to classify benchmarks but to annotate training data directly.

3. Dataset composition and pipeline coverage

The released resource covers 10 datasets across the main LLM stages. The paper explicitly analyzes CulturaX for pretraining; Dolci Instruct SFT and Aya Dataset for fine-tuning / SFT; UltraFeedback and PRISM for alignment; OpenThoughts for reasoning; and ShareLM as a real-world usage dataset. It also states that the release covers culture-centric benchmark datasets in addition to these pipeline datasets (Sahu et al., 11 Jun 2026).

The authors report the proportion of culturally tagged samples for several datasets, and these values are central to the culture-funnel argument.

Dataset	Pipeline stage	Culturally tagged samples
CulturaX	Pretraining	64.9%
Dolci Instruct SFT	Fine-tuning / SFT	12.0%
Aya Dataset	Fine-tuning / SFT	67.99%
UltraFeedback	Alignment	17.9%
PRISM	Alignment	50.30%
OpenThoughts	Reasoning	0.76%
ShareLM	Real-world chat use	23.92%

These figures support the paper’s core claim that pretraining is culturally dense, post-training data is much less culturally explicit, and reasoning datasets show the lowest cultural density. Among the stages, OpenThoughts is singled out as having the smallest cultural proportion among the major pipeline datasets. The authors interpret this as evidence that reasoning optimization pulls the pipeline toward culturally sparse synthetic or technical tasks.

The paper also reports an internal imbalance among cultural subdimensions. The most common tags are CultureAsKnowledge and GeneralCulture, while CultureAsDynamics, CultureAsPreference, and CultureAsBias are less represented. This indicates that current data is better suited to supporting fact-like cultural recall than social, normative, or dynamic cultural understanding. A plausible implication is that post-training corpora may preserve artifacts of culture more easily than they preserve culturally situated reasoning.

4. Empirical findings on geography, domains, and task intent

The geographic analysis shows that cultural content is long-tailed geographically. A small number of geolocations dominate culturally tagged examples in both pretraining and SFT. The leading locations repeatedly include India, the United States, China, and several European countries. CulturaX is described as heavily dominated by Asian and European geolocations, with very limited representation for South America, Africa, and North America in the top-ranked geolocations, despite its multilingual breadth. Dolci SFT is said to be more geographically diverse than CulturaX in cultural samples, but still imbalanced (Sahu et al., 11 Jun 2026).

This analysis yields one of the paper’s most explicit negative results: adding more languages tends to increase geolocation diversity, but does not necessarily increase the overall percentage of cultural content. In other words, multilingual scaling broadens where cultural material comes from, but not necessarily how much explicitly cultural material survives in the pipeline.

Task-intent analysis shows that cultural content is unevenly distributed across user goals. The tasks with the most cultural content are Translation, Local information lookup, Message writing / communication, and Creative writing. The tasks with the least cultural content are Math, Coding, Technical help, and Medical questions. The paper also reports a survey of 81 participants, finding that users most want cultural awareness in creative writing, translation, and email/message writing. The authors note that this roughly matches the tasks that already contain more culture in training data, while adding that even technical and medical tasks may still benefit from more cultural grounding.

Domain analysis is consistent with the same pattern. Pretraining retains strong cultural presence in Humanities & Arts, Social Sciences, and general domains, whereas post-training becomes more concentrated in Math, Code, Science, and Technology. The paper presents domain specialization as a major mechanism behind the funnel: technical optimization objectives increasingly dominate the later pipeline and thereby compress cultural diversity.

5. Marker-based interventions and benchmark effects

CultureMarkers is not only descriptive. The paper uses its tags as training-time metadata and evaluates two interventions. The first is Cultural SFT, which fine-tunes on only culture-rich examples selected using the tags. The second is marker-augmented fine-tuning, which keeps the full dataset but appends or prepends the tags as explicit markers, following the paper’s extension of “treasure marking”. The marker-augmentation setup uses dataset-wide dropout: 0.5 and per-sample marker dropout: 0.5, with the stated aim of encouraging the model to learn the markers (Sahu et al., 11 Jun 2026).

The empirical results distinguish sharply between filtering and augmentation. Cultural SFT yields only +0.2 points on NormAd, remains roughly comparable on BLEnD, and hurts knowledge-focused benchmarks and math benchmarks. The paper interprets this as evidence of capacity tradeoffs and forgetting when one simply filters for cultural density. By contrast, marker-augmented fine-tuning produces substantially stronger gains: NormAd: +8.0 points and BBQ: +6.0 points, while also improving or preserving general multilingual performance better than Cultural SFT. The appendix further reports that marker augmentation helps particularly in non-English languages on MGSM and in Europe, West Asia, Asia-Pacific, and the Americas on NormAd.

These results support the paper’s practical claim that explicit metadata can improve how models learn cultural structure. The intervention is not to remove technical data from the post-training distribution, but to preserve its breadth while making cultural content legible to the model. This suggests that CultureMarkers functions both as an audit instrument and as a mechanism for data-conditioned alignment.

6. Position within cultural evaluation research

CultureMarkers belongs to a broader shift in cultural NLP, but it addresses a different layer of the stack. Benchmark-centered work has primarily evaluated model behavior after training. CURE argues that thin evaluation systematically overestimates cultural competence and proposes thick, scenario-based assessment with Coverage, Specificity, Connotation, and Coherence as complementary metrics (Vo et al., 15 Nov 2025). XCR-Bench treats culture-specific items (CSIs) as the operational unit for cultural markers and evaluates CSI Identification, CSI Prediction, and CSI Adaptation in parallel sentence pairs (Kabir et al., 20 Jan 2026). ExCAM shifts evaluation toward free text by introducing a reference-free metric that identifies, rates and explains cultural errors in instruction-output pairs (Leiter et al., 28 May 2026).

Against that background, CultureMarkers targets the upstream precondition for those evaluations. Its claim is not that benchmarking is unnecessary, but that benchmark outcomes are constrained by what the training data pipeline retains. This suggests a division of labor across recent work. Anthropological critiques argue that culture should not be reduced to static facts or homogeneous national profiles (AlKhamissi et al., 7 Oct 2025). Benchmark frameworks such as CURE and XCR-Bench test whether models can reason about cultural meaning in context (Vo et al., 15 Nov 2025, Kabir et al., 20 Jan 2026). ExCAM measures culturally erroneous output at the level of spans, severities, and explanations (Leiter et al., 28 May 2026). CultureMarkers, by contrast, asks whether the training data even contains the cultural substrate required for those capabilities to emerge robustly (Sahu et al., 11 Jun 2026).

The main implication is methodological. Cultural alignment should not be treated as a purely inference-time or evaluation-time problem. The paper argues instead for changes in data design across the full pipeline: curating data with culture in mind, not assuming language coverage implies cultural coverage, balancing domains so technical data does not swamp culturally grounded data, preserving long-tail geographies, and using explicit markers to make culture learnable during training. In that sense, CultureMarkers reframes cultural capability as a property of the data pipeline itself: culture can be evaluated downstream, but it must be represented upstream.