Updesh: Culturally Grounded Indic Dataset

• Updesh is a culturally contextualized synthetic dataset comprising 9.5M examples across 13 Indic languages, addressing data scarcity and cultural misalignment.
• It uses a dual pipeline combining bottom-up generation from local Wikipedia artifacts with top-down translation to preserve native idioms and cultural relevance.
• The dataset enhances NLU and NLG performance for low-resource Indic languages, showing significant improvements on rigorous benchmark evaluations.

Updesh is a high-quality, large-scale synthetic instruction-following dataset comprising 9.5 million data points across 13 Indian languages. It embodies a comprehensive initiative to address the dual challenges of data scarcity and cultural misalignment in multilingual AI, particularly for low- and medium-resource Indic languages. Updesh emphasizes culturally contextualized data generation, supporting a diverse range of reasoning and generative tasks, with special attention to long-context and multi-turn instruction-following capabilities. The methodology integrates both top-down and bottom-up paradigms—combining large-scale translation of language-agnostic content with context-grounded synthetic generation based on language-specific Wikipedia artifacts—leading to significant improvements in downstream natural language understanding (NLU) and generation (NLG) for under-served languages (Chitale et al., 25 Sep 2025).

1. Motivation and Foundational Challenges

The pre-training imbalance inherent in standard LLM corpora—dominated by high-resource languages such as English and Chinese—results in substantial under-representation of most Indian languages. Compounding this, existing corpora often reflect Western cultural norms, idioms, and epistemologies, which impedes models from capturing local knowledge, cultural artifacts, or discourse styles relevant to Indian contexts. Synthetic data augmentation, if linguistically and culturally grounded, provides a viable pathway to redress these gaps by expanding available corpora in ways that encode native linguistic phenomena and contextually accurate socio-cultural information. Cultural grounding—e.g., by leveraging region- and language-specific Wikipedia content on festivals, cuisine, and folklore—ensures that generated tasks and answers resonate with native speakers and capture regionally specific factual and cultural knowledge (Chitale et al., 25 Sep 2025).

2. Construction Methodology and Dataset Composition

Updesh employs a dual-pipeline construction strategy. The bottom-up approach generates new data directly in the target language, while the top-down paradigm translates existing English corpora:

• Bottom-up Generation: For reasoning tasks, approximately 450,000 English examples from OrcaAgent-Instruct and OrcaMath are translated to each target language using Llama-3.1-405B-Instruct, with parameters top$_p = 0.95$ and $T = 1.0$, followed by back-translation validation. For culturally grounded generative tasks, the Qwen3-235B-A22B model is prompted over local Wikipedia articles classified under "Culture of India" (54 pan-Indian categories, 19,143 artifacts) and state-specific categories (37,227 artifacts). Sampled artifacts ($\approx$26.8K) are used to prompt the LLM for multi-hop questions and answers, resulting in approximately 7.5 million synthetic examples before filtering.
• Top-down Translation: Prior paradigms focused on translating English instruction-response pairs, often introducing translationese artifacts, loss of idiomatic specificity, and insufficient cultural relevance.
• Final Composition: After automated filtering (IndicLID $\geq$ 0.75; repetition ratio $\leq$ 0.75), Updesh contains 9.5 million examples across 13 Indic languages: Assamese, Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Odia, Punjabi, Tamil, Telugu, and Urdu. The dataset consists of two complementary subsets:

Subset	Number Examples	Tasks Included
Reasoning	$\approx$4.7M	Analytical Reasoning, MCQs, Fermi Guesstimation, Few-Shot CoT, Brain Teasers, Text Classification, RC, Math WP
Generative	$\approx$4.8M	Logical Inference, Multi-Hop QA, Causal Reasoning, Creative Writing, Multi-Turn Dialogue, Summarization, MT, CMR

3. Instruction Types, Task Design, and Prompt Schema

Updesh spans a suite of both reasoning and generative tasks to foster broad instruction-following competence, long-context comprehension, and multi-turn dialogue capabilities.

• Reasoning: One-phase prompts for analytical reasoning, multiple-choice questions (MCQ, with detailed explanations), Fermi guesstimation, few-shot chain-of-thought (4–5 in-context demonstrations), brain teasers, text classification, reading comprehension (passages up to 2K tokens), and math word problems.
• Generative: Frequently two-phase prompts for logical inference, multi-hop question answering (QA), causal reasoning, creative writing, multi-turn persona-driven dialogue (3–5 rounds), summarization of long documents ($\geq$1K tokens), machine translation preserving cultural references, and cultural multi-hop reasoning spanning multiple artifacts.
• Prompt Schema Example: Multi-turn dialogue specifications formalize as:

$$U_1, A_1, U_2, A_2, \ldots, U_n \rightarrow \text{generate } A_{n+1}$$

with the constraint:

$$\arg\max_{A_{n+1}} \; P(A_{n+1} \mid U_{1:n}, A_{1:n}, \text{persona})$$

preserving persona and topic coherence.

4. Quality Assurance and Evaluation

Quality assurance integrates automated filtering and extensive human annotation:

• Automated Filtering: Language identification (IndicLID $T = 1.0$0 0.75) and repetition ratio ($T = 1.0$1 0.75) remove $T = 1.0$2 of samples overall (up to $T = 1.0$3 for Assamese misclassified as Bengali). Back-translation fidelity using Llama-3.3-70B yields high ChrF scores (e.g., Hindi: 84–87, Bengali: 87–88, Urdu: 85). Average ChrF for reasoning tasks is 75–85. BLEU/ROUGE/METEOR/F1 are not reported.
• Human Annotation: Fifteen native speakers assessed approximately 10,000 samples (five languages × four generative categories × 100 samples; plus 400 reasoning per language), rating on 3-point scales across Instruction Adherence, Fluency, Coherence, Cultural Relevance, and Toxicity. Only 27 of 10,000 assessments (0.27%) received the lowest score, indicating high quality. Inter-annotator agreement exceeded 90% on objective measures and was 60–70% on culturally nuanced, subjective metrics. LLM-as-judge (GPT-4o) agreement with human raters varied, underscoring the necessity of human oversight for cultural criteria (Chitale et al., 25 Sep 2025).

5. Downstream Fine-Tuning and Empirical Results

Fine-tuning experiments leveraged Llama-3.1-8B-HF and Phi4-14B using the Axolotl framework (sequence length 65,536, batch size 8,192, 3 epochs, AdamW optimizer, and BF16 with flash attention). Baseline comparisons included Bactrian-X (3.4M samples, 10 languages), IndicAlign (7.3M after downsampling), and Aya-Collection (7M balanced samples). Performance was measured across 15 benchmark datasets, covering NLU (MMLU, MMLU-Indic, MILU, ARC-Indic, BoolQ-Indic, TVQA-Indic, BeleBele, INCLUDE, Global MMLU), NLG (XSum, Flores En→XX, Flores XX→En, IN22-Conv-Doc), and instruction-following (mIFEval, mIFBench).

• NLU: Phi4+Updesh achieved or surpassed all baselines (e.g., MMLU-I: 0.49 vs 0.45, MILU: 0.54 vs 0.53, BoolQ-I: 0.81 vs 0.76, BeleBele: 0.74 vs 0.61, Global MMLU: 0.57 vs 0.55). Llama+Updesh was competitive on reading comprehension and instruction-following tasks.
• NLG: Llama+Updesh dramatically exceeded all baselines on XSum (ChrF: 17.4 vs 0.26–9.3), Flores En→XX (32.6 vs 22.6–25.2), Flores XX→En (46.8 vs 0.5–2.3), and IN22-Conv (21.3 vs 6.7–14.1). Phi4+Updesh also demonstrated strong translation performance (Flores XX→En: ChrF=53.3).
• Instruction Following: Llama+Updesh minimized catastrophic-forgetting on mIFEval/mIFBench, with Phi4+Updesh exhibiting modest accuracy gains on mIFBench while maintaining mIFEval performance.
• Resource-tier Analysis: Grouping languages by Joshi et al. resource class (0–5), Updesh yielded the largest relative accuracy and ChrF gains in low/mid-resource language tiers, narrowing performance gaps to high-resource languages by as much as 15–20% (Chitale et al., 25 Sep 2025).

6. Key Lessons and Limitations

Empirical results substantiate that culturally grounded, context-aware synthetic datasets enhance generative capabilities—summarization, translation, and long-context dialogue—far beyond those derived from purely translated or template-based corpora. Multiple-choice NLU outcomes are more mixed, attributed to task format mismatches (Updesh focuses on long-answer MCQ) and possible saturation for larger pre-trained models. Automated LLM-based evaluation remains unreliable for subjective, culturally nuanced criteria, reinforcing the critical role of human judgment. The principal limitations are residual hallucinations in generative outputs and inconsistent quality in very low-resource languages (e.g., Odia, Assamese), where backtranslation flagged more anomalies. A plausible implication is that subsequent efforts could benefit from integrating native-speaker seed data and iterative self-curation.

7. Methodological Implications and Future Directions

The Updesh approach—synthesizing top-down translated, language-agnostic reasoning data with bottom-up, Wikipedia-grounded culturally relevant data—constitutes a generalizable framework for under-served language families beyond the Indic context. Key methodological features include:

• Direct, large-scale, language-targeted generation with open-source LLMs
• Systematic extraction of cultural taxonomies from MediaWiki APIs to ground tasks in regional artifacts and values
• Automation-augmented, multi-stage filtering combining language ID, repetition checks, and targeted human audits
• Multifaceted evaluation blending automated metrics, human review, and diverse downstream benchmarks

This suggests that linguistically and culturally faithful synthetic data initiatives like Updesh can significantly mitigate resource disparities, promoting more inclusive and equitable advances in multilingual AI (Chitale et al., 25 Sep 2025).