L3Cube-MahaParaphrase Dataset

• L3Cube-MahaParaphrase Dataset is a balanced Marathi paraphrase detection corpus containing 8,000 sentence pairs equally split between paraphrase and non-paraphrase labels.
• It combines cosine similarity and back-translation methods to generate diverse and controlled candidate sentence pairs with detailed lexical overlap bucketing.
• Fine-tuned transformer models on this dataset achieve notable performance, with MahaBERT attaining an F1-macro score of 88.70% for binary paraphrase classification.

The L3Cube-MahaParaphrase Dataset, introduced in "MahaParaphrase: A Marathi Paraphrase Detection Corpus and BERT-based Models" (Jadhav et al., 24 Aug 2025), is a Marathi paraphrase-detection corpus for a low resource Indic language. It contains 8,000 sentence pairs, each annotated by human experts as either Paraphrase (P) or Non-paraphrase (NP), with an overall P:NP ratio of 1:1. The accompanying study also reports results for standard transformer-based BERT models fine-tuned for binary paraphrase classification, and publicly releases the dataset and model artifacts.

1. Corpus definition and scope

MahaParaphrase is designed as a balanced binary classification resource for Marathi sentence-pair semantics. The corpus comprises 4,000 Paraphrase pairs and 4,000 Non-paraphrase pairs. Average sentence lengths are reported separately for the two classes: for P, $S_1 = 10.43$ words and $S_2 = 9.99$ words, with $\Delta = 1.45$; for NP, $S_1 = 9.52$ words and $S_2 = 11.22$ words, with $\Delta = 3.36$. The sentence-length histogram peaks in the $[5\text{–}15]$-word range for both P and NP.

The dataset is situated in the context of Marathi NLP, where rich morphological and syntactic variations, diverse scripts, and limited availability of annotated data make supervised semantic tasks difficult. Within that setting, a manually verified paraphrase corpus directly supports paraphrase detection and related language understanding tasks.

Statistic	Value
Total sentence pairs	8,000
Paraphrase pairs	4,000
Non-paraphrase pairs	4,000
Class ratio	1:1
Buckets per class	5
Pairs per bucket	800

This balanced design makes the resource suitable for controlled binary evaluation. A plausible implication is that the corpus was intended not only as a benchmark for classification accuracy, but also as a structured testbed for lexical-overlap-sensitive analysis.

2. Construction pipeline

All raw sentences were drawn from L3Cube’s MahaCorpus (Joshi 2022), described as a large-scale Marathi monolingual collection scraped mostly from Maharashtrian news websites and spanning multiple domains, including news, sentiment, and hate speech. A random sample of 1 million sentences was extracted from MahaCorpus as candidate material (Jadhav et al., 24 Aug 2025).

Candidate pair generation used two complementary approaches, each contributing 50% of the final pool. The first was cosine-similarity based pairing. Sentence embeddings were generated using MahaSBERT, a sentence-BERT model fine-tuned on Marathi data. Every possible sentence pair was scored by cosine similarity. Pairs with $\mathrm{CS} \ge 0.80$ and $\mathrm{CS} \le 0.99$ were labelled Paraphrase, while pairs with $\mathrm{CS} < 0.80$ were labelled Non-Paraphrase.

The second approach was back-translation. Each source sentence $S_2 = 9.99$0 was translated to English as $S_2 = 9.99$1 and then back-translated to Marathi as $S_2 = 9.99$2 via Google Translate. The pair $S_2 = 9.99$3 was retained as a paraphrase candidate only if $S_2 = 9.99$4, with cosine similarity computed using MahaSBERT for Marathi–Marathi and IndicSBERT for Marathi–English. Pairs with $S_2 = 9.99$5 or $S_2 = 9.99$6 were discarded to avoid semantic drift or trivial identity.

This hybrid construction procedure combines synthetic and comparatively more natural candidate generation. This suggests that the resource was engineered to include both semantically close reformulations and harder negative cases, rather than relying on a single retrieval heuristic.

3. Annotation, bucketing, and released schema

Four native-Marathi annotators manually verified all candidate pairs, correcting misclassifications and translation artifacts. The annotation schema contains only two labels, P and NP. The paper does not report any inter-annotator agreement, such as Cohen’s $S_2 = 9.99$7 or Fleiss’s $S_2 = 9.99$8, and it does not describe an adjudication procedure beyond majority verification.

For fine-grained analysis, both the P and NP subsets were divided into five equal buckets, $S_2 = 9.99$9, based on word-overlap percentage. The buckets are defined as $\Delta = 1.45$0, $\Delta = 1.45$1, continuing up to $\Delta = 1.45$2. Word overlap is defined as

$\Delta = 1.45$3

This yields 10 total buckets, five for P and five for NP, each containing 800 sentence pairs (Jadhav et al., 24 Aug 2025).

Although the paper does not explicitly list the file schema, the publicly released CSV and JSON are described as following a straightforward sentence-pair format. The CSV contains sentence1, sentence2, label, and bucket; the JSON contains id, sentence1, sentence2, label, and bucket.

Representative examples illustrate why lexical overlap is treated as an analytic dimension rather than a label surrogate. A low-overlap paraphrase in bucket $\Delta = 1.45$4 is:

S₁: “मोठा मोबाइल कंपन्‍या विविध फीचर्स वापरक्‍यांसाठी आणत आहेत ।” S₂: “मोबाइल निर्माता कंपन्‍या ग्राहकांसाठी नवीन वैशिष्‍ट्ये आणते आहेत ।” Overlap $\Delta = 1.45$5

A high-overlap non-paraphrase in bucket $\Delta = 1.45$6 is:

S₁: “दोन वर्षांत दोन कोटी लोकांनी मोबाईल बुक केले आहेत ।” S₂: “मोठ्या संख्येने लोकांनी दोन वर्षांत मोबाईल खरेदी केली आहे ।” Overlap $\Delta = 1.45$7

These examples directly counter a common simplification in paraphrase detection: high lexical overlap does not guarantee semantic equivalence, and low overlap does not preclude paraphrase.

4. Preprocessing, model inputs, and fine-tuning regime

The paper does not detail extensive text normalization beyond what is inherent in the BERT tokenizers. No additional Unicode normalization or diacritic stripping is reported. Tokenization is delegated to the corresponding model’s WordPiece or SentencePiece tokenizer, including Devanagari-aware tokenization in MahaBERT. No explicit handling of code-mixed English–Marathi tokens is reported beyond their inclusion in the raw sentences.

Five transformer checkpoints were fine-tuned on the L3Cube-MahaParaphrase train set: mBERT (bert-base-multilingual-cased), MuRIL, IndicBERT (MLM-only), IndicBERT (MLM+TLM), and MahaBERT. IndicBERT (MLM-only) is described as ALBERT-based and trained on 12 Indic languages, while IndicBERT (MLM+TLM) adds translation-language modeling. MahaBERT is described as L3Cube’s Marathi-refined BERT (Jadhav et al., 24 Aug 2025).

The common input format is:

$S_1 = 9.52$3

The task objective is binary classification of P versus NP with cross-entropy loss. The consolidated overview lists likely hyperparameters rather than explicitly reported ones: learning rate $\Delta = 1.45$8, batch size 16 or 32, and 3–5 epochs, with AdamW, linear warm-up, and weight decay. Validation via early stopping on dev set accuracy/F1 is mentioned as not explicitly reported.

The absence of fully specified hyperparameters is methodologically important. It limits exact training reproducibility at the level of optimization settings, even though the dataset and fine-tuned artifacts are publicly shared.

5. Evaluation protocol and reported performance

The study reports a single “Score,” interpreted in the consolidated overview as F1-macro over both classes. Standard metrics are also given:

$\Delta = 1.45$9

$S_1 = 9.52$0

$S_1 = 9.52$1

$S_1 = 9.52$2

Reported model performance, expressed as F1-macro in percent, is as follows (Jadhav et al., 24 Aug 2025):

Model	F1-macro
MahaBERT	88.70
IndicBERT (MLM+TLM)	87.10
MuRIL	86.90
IndicBERT (MLM only)	85.90
mBERT	84.59

MahaBERT outperforms all baselines by approximately 1–4 points. No error analysis or per-bucket breakdown is provided. Because the corpus is explicitly bucketed by lexical overlap, a plausible implication is that the reported aggregate scores may mask materially different behaviors across low-overlap and high-overlap regions of the data.

6. Limitations, applications, and research trajectory

Several limitations are explicitly noted. The corpus size, at 8,000 pairs, is small relative to high-resource languages. Some code-mixed pairs introduce noise for strictly Marathi models. The paper does not report inter-annotator agreement, train/dev/test splits, or error analysis (Jadhav et al., 24 Aug 2025).

The resource is positioned for paraphrase detection in Marathi, with applications to question answering, style transfer, semantic parsing, and data augmentation. Additional potential applications listed in the consolidated overview include plagiarism detection and semantic clustering in Marathi text. Because the corpus is manually verified and bucketed by lexical overlap, it is also suited to controlled evaluation of lexical versus semantic generalization.

The publicly released artifacts include the dataset on HuggingFace, code and CSV/JSON files on GitHub, and a fine-tuned MahaParaphrase-BERT model on HuggingFace. The public release of both the bucketed corpus and the fine-tuned models supports reproducible research on Marathi paraphrase phenomena.

Future directions named in the source include expanding dataset scale and domain coverage, introducing graded paraphrase labels or multi-way annotations, evaluating larger LLMs and non-BERT architectures, and performing bucket-wise analysis to understand lexical versus semantic generalization. Taken together, these directions indicate that MahaParaphrase functions both as a benchmark and as an initial substrate for more detailed Marathi semantic similarity research.