Sangraha: Curated Computational Resources

Updated 14 July 2026

Sangraha is a computational concept that curates and aggregates diverse Sanskrit and Indic data into structured collections for analysis.
It underpins systems like IndicLLMSuite, Mitrasamgraha, and Sangrahaka by organizing raw texts, translations, and knowledge graphs through rigorous cleaning and deduplication.
Its methodologies emphasize structured aggregation, metadata enrichment, and post-correction, enabling robust pre-training, translation, and restoration processes.

Sangraha, in recent computational research on Sanskrit and adjacent Indic domains, denotes neither a single dataset nor a single algorithmic paradigm. It appears instead as a recurring organizational concept for curated collections, summary-oriented representations, and systematized interfaces that assemble, normalize, query, or restore structurally complex material. Current usage spans LLM pre-training corpora, Sanskrit–English machine-translation resources, knowledge-graph tooling, raw-text comparison methods, and structurally constrained correction systems for meter and music (Khan et al., 2024, Nehrdich et al., 12 Jan 2026, Terdalkar et al., 2021, Li, 2023).

1. Contemporary research usage

The recent literature does not support treating Sangraha as a single standardized technical object. Rather, the term and close cognates are distributed across several artifact classes: a pre-training resource suite in IndicLLMSuite, a Sanskrit–English bitext dataset in Mitrasamgraha, a knowledge-graph annotation and querying tool in Sangrahaka, and a summarization-oriented motivation in n-aksara modeling. In ShrutiSense, the term is also used in the sense of preserving the essential “collection/system” of shruti positions. This suggests a contemporary technical usage centered on collection, condensation, and structured retention rather than on one fixed architecture (Khan et al., 2024, Nehrdich et al., 12 Jan 2026, Terdalkar et al., 2021, Li, 2023, Ghosh et al., 2 Aug 2025).

Name	Function	Scale or scope
IndicLLMSuite “Sangraha”	Pre-training resources for Indic LLMs	22 languages; 251B tokens
Mitrasamgraha	Classical Sanskrit machine translation dataset	391,548 bitext pairs
Sangrahaka	Annotation and querying of knowledge graphs	Web-based; language and corpus agnostic
n-aksara modeling	Raw-text modeling linked to Sangraha as summarization	105 commentaries; 10 centuries; 9 languages
ShrutiSense	Microtonal correction and completion	22 shrutis; five ragas

A common misconception is to read these occurrences as mere naming coincidence. The published systems indicate something more structured: Sangraha regularly designates a resource that gathers dispersed material into a form suitable for computation, inspection, or restoration. The resulting objects differ sharply in modality—text, graph, or pitch sequence—but they share an emphasis on controlled aggregation and subsequent access or correction.

2. Sangraha at pre-training scale

In IndicLLMSuite, “Sangraha” denotes the pre-training component of a resource suite for Indic LLM development. It covers 22 scheduled Indian languages, contains 251B tokens, and is partitioned into Sangraha Verified at 64B, Sangraha Synthetic at 162B, and Sangraha Unverified at 24B. Data acquisition spans web text, digitized PDFs, speech transcriptions, synthetic translations and Romanized variants, supervised fine-tuning conversations, and toxic prompt-response datasets; the suite also includes 74.8M instruction-response pairs and 123,000 toxicity-alignment pairs (Khan et al., 2024).

Its curation logic is explicit and pipeline-driven. The Setu pipeline comprises document preparation, cleaning and analysis, flagging and filtering, and deduplication. Language identification uses an ensemble of IndicLID, cld3, and NLLB. Deduplication uses MinHashLSH with Jaccard similarity of at least 0.7 on 5-grams, and perplexity-based cleaning is performed with a 5-gram Kneser-Ney LLM trained on a 200k sample from Sangraha Verified for each language. The suite is released with permissive open-source licensing, and its stated purpose is not only to provide data but to offer a reproducible blueprint for extending such efforts to other languages (Khan et al., 2024).

The downstream utility of Sangraha-scale collection is visible in ByT5-Sanskrit. That model sources its raw Sanskrit data mainly from the IndicLLMSuite Sangraha dataset, described there as over 5.1B characters. It further pretrains ByT5-base for 100,000 steps and reports strong results across segmentation, dependency parsing, and OCR post-correction. On Sanskrit word segmentation, ByT5-Sanskrit reaches 90.11 on DCS 2018 and 94.29 on Hackathon; on OCR post-correction it reports 2.69 CER and 20.03 WER; and for Vedic dependency parsing it improves UAS and LAS beyond prior biaffine baselines. The evidence here is methodological as much as empirical: large-scale curated collection becomes actionable when paired with byte-level modeling that tolerates morphology, sandhi, and noisy orthography (Nehrdich et al., 2024).

3. Mitrasamgraha and parallel collection for translation

Mitrasamgraha instantiates Sangraha as a classical Sanskrit machine-translation dataset. It contains 391,548 Sanskrit–English bitext pairs, a validation set of 5,587 post-corrected pairs, and a test set of 5,552 post-corrected pairs. Its temporal span exceeds three millennia, organized into Vedic, Epic, Classical, and Medieval bins, and its domain coverage includes epic narratives, poetry, Purāṇa, religious scriptures, treatises, and Vedic texts. Each text carries document-level metadata including title, author, time period, genre or domain, category, and license, and the corpus draws on more than 40 human translators (Nehrdich et al., 12 Jan 2026).

The construction workflow is equally central to the resource’s identity. It proceeds through text identification with domain experts, sourcing via scraping or OCR, manual cleaning, sentence segmentation adapted per text, automatic sentence alignment with BertAlign, and manual validation. In the reported alignment benchmark, BertAlign outperforms VecAlign, with sentence-level $F_S$ ranging from 70.15 to 97.12 across sample domains. The dataset thereby operationalizes Sangraha not as undifferentiated accumulation but as curation with philological and alignment metadata attached (Nehrdich et al., 12 Jan 2026).

Benchmarking on this resource shows that collection quality alone does not eliminate Sanskrit–English MT difficulty. BLEURT and GEMBA or GEMBA* correlate much more strongly with expert human evaluation than BLEU or chrF, with average Pearson correlations of 0.3599 for BLEURT, 0.3672 for GEMBA*, and 0.3837 for GEMBA, against 0.0963 for BLEU and 0.2060 for chrF. Fine-tuning on Mitrasamgraha yields large gains: NLLB 600M rises to 35.59 chrF, 52.88 BLEURT, and 71.07 GEMBA; NLLB 3.3B reaches 36.06, 53.89, and 75.12; Gemma2 9B reaches 38.36, 55.10, and 83.34. At the same time, the paper explicitly identifies persistent challenges in sandhi, compounding, free word order, poetic and philosophical language, multi-layered metaphor, and asymmetric or noisy alignment. The Sangraha here is comprehensive, but not exhaustive with respect to translational difficulty (Nehrdich et al., 12 Jan 2026).

4. Sangrahaka and knowledge-graph infrastructure

Sangrahaka is a web-based annotation and querying tool for knowledge graphs. It annotates entities and relationships from text corpora, constructs a KG, and supports querying through templatized natural-language questions mapped to Cypher. The framework is language and corpus agnostic but tunable to the special needs of a specific language or corpus, and it is built on a stack that includes Python 3.8, Flask 1.1.2, Neo4j Community Server 4.2.1, SQLite 3.35.4, and a frontend using HTML5, JavaScript, Bootstrap 4.6, vis.js, and Jinja2 (Terdalkar et al., 2021).

Its workflow is role-structured. Admin uploads the corpus and designs the ontology; Annotators mark entities and relationships; Curators resolve conflicting annotations; graph-generation scripts consolidate annotation data into a Neo4j KG; and Queriers access the graph through natural-language templates or raw Cypher. Results are shown both as interactive graphs and tables and can be exported as CSV, JSON, or text. The data model distinguishes entity annotation and relationship annotation, with lemmas functioning as unique identifiers for graph nodes in the import process (Terdalkar et al., 2021).

What makes Sangrahaka specifically relevant to the broader Sangraha pattern is that it turns annotated corpora into a queryable collected structure rather than stopping at mark-up. It also foregrounds operational resilience: annotation actions are staged as Prepare and Confirm, prepared annotations persist in browser localStorage, and the interface is fault tolerant on both client and server side. The paper reports an objective benchmark adapted from Neves et al. (2021) in which Sangrahaka obtains the highest composite score, 0.82, against 0.78 for FLAT, 0.74 for WebAnno, and 0.70 for BRAT. Two deployments are described: BPN, a Sanskrit Ayurveda corpus with 180 lines, 25 node types, 30 relation types, 602 entity annotations, and 778 relation annotations; and VR, an English Ramayana corpus with 17,655 lines, 107 node types, 132 relation types, 1,810 entity annotations, and 2,087 relation annotations (Terdalkar et al., 2021).

5. Sangraha as summarization and raw-text modeling

The n-aksara model addresses a different but closely related problem: how to represent Sanskrit and Sanskrit-adjacent texts without relying on unstable word segmentation. An aksara is defined as a pronounceable unit ending with a vowel, anusvāra, or visarga, and an n-aksara is a contiguous sequence of $n$ aksaras. This representation avoids the need for sandhi resolution and compound splitting, permitting analysis directly on raw text. It is also described as script-robust and usable for Sanskrit-adjacent material such as a Tamil commentary on a Sanskrit text (Li, 2023).

The paper formalizes text comparison through set-based similarity metrics. For texts represented as sets of n-aksaras, it uses the Jaccard Index

$J(A,B) = \frac{|A \cap B|}{|A \cup B|}$

and the Dice Coefficient

$D(A,B) = \frac{2|A \cap B|}{|A| + |B|}.$

It also allows skip-grams and fuzzy matching, for example by ignoring some vowels or harmonizing orthographic variants, to better accommodate inflectional endings and cross-language comparison (Li, 2023).

As a test case, the method models 105 commentaries on Amarakosa 1.0.1 across 10 centuries and 9 languages as sets of 2-, 3-, 4-, and 5-aksaras. A similarity graph constructed with Dice coefficients as edge weights and summarized by a minimum spanning tree reveals clustering among witnesses of the same commentary, among texts in the same geographic or language family, and across languages through reused Sanskrit phrases. The paper also links this directly to Sangraha in the sense of automatic summarization: since summarization depends on identifying salient and recurring textual units, n-aksaras provide a granularity that remains meaningful even when words and compounds are ambiguous, variable, or fused. In this usage, Sangraha is less a database than a method of stable reduction under philological uncertainty (Li, 2023).

6. Structural correction, restoration, and preservation

A further extension of the Sangraha pattern appears in systems that reconstruct structured order from corrupted or incomplete inputs. Chandojñānam is a web-based Sanskrit meter identification and utilization system. It accepts typed or pasted Sanskrit text in any transliteration supported by indic-transliteration, converts all text internally to Devanagari, supports OCR from images through either Google Drive OCR or Tesseract OCR, and processes entire uploaded text files in either line mode or verse mode. Its metrical database stores “lg-signatures” of laghu and guru sequences in dictionaries for single and multiple pāda matching and includes regex-based definitions for complex meters such as Anuṣṭubh (Terdalkar et al., 2022).

Its processing pipeline is transliteration to pāda split to syllabification to laghu-guru marking, followed by lg-signature extraction, direct matching, and fuzzy matching. If no exact match is found, the system computes Levenshtein edit distance against signatures in the database, returns top- $k$ probable meters with $k=10$ by default, and issues edit instructions such as insert, delete, or replace, sometimes specifying syllable or vowel substitution. The reported application is corpus correction: Chandojñānam identifies anomalous lines, aids post-OCR review, and significantly outperforms previous tools in recovering intended meters from noisy or corrupted texts, reaching 98.2% correct on test corpora. Its current scope is limited to Varnavrtta meters, not Mātrāvrtta, and the meter database covers about 200 meters (Terdalkar et al., 2022).

ShrutiSense performs an analogous operation in symbolic music processing. It models the 22-shruti system, corrects westernized or corrupted pitch sequences, and completes melodic sequences with missing values by combining a Shruti-aware finite-state transducer for correction with a grammar-constrained Shruti hidden Markov model for completion. On simulated data across five ragas, the FST model achieves 91.3% shruti classification accuracy for correction tasks, maintains 90.7–91.8% across ragas, and remains robust under pitch noise up to $\pm 50$ cents. The paper describes this as preserving the essential sangraha, that is, the collection or system, of shruti positions. Across meter and music alike, the shared computational principle is restoration of a constrained structural inventory under noisy observation (Ghosh et al., 2 Aug 2025).

7. Limits, misconceptions, and research directions

The collected evidence indicates that Sangraha should not be confused with simple scale. IndicLLMSuite emphasizes best practices for crawling, cleaning, flagging, and deduplication; Mitrasamgraha attaches temporal and domain metadata and uses post-corrected validation and test splits; Sangrahaka builds ontology-aware and role-aware interfaces; Chandojñānam combines exact and fuzzy metrical matching; and ShrutiSense hard-codes raga grammar into correction and completion. The recurring design lesson is that collection without structural control is insufficient for Sanskrit and related Indic domains (Khan et al., 2024, Nehrdich et al., 12 Jan 2026, Terdalkar et al., 2021, Terdalkar et al., 2022, Ghosh et al., 2 Aug 2025).

The limitations are correspondingly domain specific. ByT5-Sanskrit does not currently distinguish all homonyms. Mitrasamgraha reports significant challenges in the translation of complex compounds, philosophical concepts, and multi-layered metaphors. Chandojñānam does not yet handle Mātrāvrtta meters and presently covers about 200 meters. Sangrahaka’s authors state that installation may be streamlined further, for example via Docker. ShrutiSense identifies future work in polyphony, ornamentation, multi-modal input, and cross-cultural systems such as maqam and Carnatic modeling (Nehrdich et al., 2024, Nehrdich et al., 12 Jan 2026, Terdalkar et al., 2022, Terdalkar et al., 2021, Ghosh et al., 2 Aug 2025).

A plausible implication is that future Sangraha-type resources will continue to combine three features: curated aggregation, explicit structural constraints, and human validation or post-correction. The current literature already shows that this combination is productive across LLM pre-training, machine translation, knowledge-graph construction, textual comparison, metrical correction, and microtonal music modeling. In that sense, Sangraha has become a cross-domain research motif for turning difficult, noisy, and heterogeneous Indic materials into computationally tractable yet structurally faithful objects.