---
title: 'MahaSBERT-STS-v2: Marathi SBERT for STS'
url: https://www.emergentmind.com/topics/mahasbert-sts-v2
type: topic
---

# MahaSBERT-STS-v2: Marathi SBERT for STS

MahaSBERT-STS-v2 is a Marathi Sentence-BERT model for sentence-level semantic similarity, implemented as a siamese/bi-encoder network and optimized for regression-style similarity scoring with cosine similarity [2508.21569]. In the most recent formulation, it denotes the model fine-tuned directly on MahaSTS, a human-annotated Marathi Sentence Textual Similarity dataset containing 16,860 sentence pairs with continuous labels from 0 to 5 [2508.21569]. Earlier L3Cube work released a Marathi STS model under the identifier `l3cube-pune/marathi-sentence-similarity-sbert`, referred to in text as “MahaSBERT-STS,” and the “v2” designation is used in later descriptions to connect that STS model to the `marathi-bert-v2` backbone or, in the 2025 work, to distinguish the MahaSTS-fine-tuned variant from the earlier NLI-only checkpoint [2304.11434] [2211.11187].

## 1. Nomenclature and model identity

The term MahaSBERT-STS-v2 has two closely related usages in the L3Cube literature. In the 2025 work “L3Cube-MahaSTS: A Marathi Sentence Similarity Dataset and Models,” it denotes a Marathi Sentence-BERT model fine-tuned specifically on MahaSTS, with the base encoder initialized from MahaSBERT, which itself had been trained on IndicXNLI using MahaBERT as the underlying backbone [2508.21569]. In the earlier L3Cube sentence-embedding literature, the exact string “MahaSBERT-STS-v2” is not used in the paper text; instead, the released STS model is “MahaSBERT-STS,” available as `l3cube-pune/marathi-sentence-similarity-sbert`, and “v2” is associated with the `marathi-bert-v2` backbone [2304.11434] [2211.11187].

This naming overlap is central to interpreting the model lineage. The earlier model family distinguishes between MahaSBERT, the NLI-only checkpoint `l3cube-pune/marathi-sentence-bert-nli`, and MahaSBERT-STS, the two-step NLI+STS model [2211.11187]. The 2025 paper adds a new task-specific stage by fine-tuning on MahaSTS, a native human-annotated Marathi STS resource rather than only translated STS data, and explicitly states that the v2 fine-tuning on MahaSTS yields notable performance gains over the NLI-only model for sentence similarity [2508.21569].

A common misconception is to treat all references to MahaSBERT-STS-v2 as pointing to a single invariant checkpoint. The record is more specific: the 2022 and 2023 papers describe the released Marathi STS model corresponding to the two-step NLI→STS recipe, whereas the 2025 paper uses the name for the MahaSTS-fine-tuned regression model and benchmarks it on the new dataset [2304.11434] [2211.11187] [2508.21569].

## 2. Architectural design and similarity formulation

MahaSBERT-STS-v2 follows the standard SBERT design. Two sentences are independently encoded into dense vectors by a transformer encoder with shared weights, and a pooling operation converts token-level representations into fixed-length sentence embeddings [2508.21569]. The underlying Marathi backbone is MahaBERT, released as `l3cube-pune/marathi-bert-v2`, which is described as a Marathi BERT derived from `google/muril-base-cased` and further fine-tuned on L3Cube-MahaCorpus and other Marathi monolingual datasets [2508.21569]. In earlier L3Cube descriptions, the backbone is identified as L3Cube-MahaBERT v2 trained or fine-tuned on 752M Marathi tokens from L3Cube-MahaCorpus and other Marathi corpora [2211.11187].

Pooling strategy is a salient design choice. The 2025 study evaluates CLS, MEAN, and MAX pooling and reports that MEAN pooling consistently performs best; MahaSBERT-STS-v2 therefore uses MEAN pooling [2508.21569]. The 2022 and 2023 papers report the same directional conclusion in slightly different terminology, describing AVG or mean pooling as superior to CLS for Marathi monolingual SBERT [2304.11434] [2211.11187]. The mean-pooled sentence embedding is described in the earlier work as
$$
e = \frac{1}{T}\sum_{t=1}^{T} h_t
$$
where $h_t$ are token representations [2211.11187].

Sentence similarity is computed using cosine similarity between the two sentence embeddings:
$$
\hat{s} = \frac{\mathbf{u}\cdot\mathbf{v}}{\|\mathbf{u}\|\,\|\mathbf{v}\|}
$$
[2508.21569]. The STS objective is CosineSimilarityLoss from Sentence-Transformers, i.e., cosine similarity–based regression to gold similarity scores [2508.21569]. The 2022 and 2023 descriptions make explicit that this loss is used after the NLI stage in the two-step pipeline, with the target typically treated as a normalized version of the gold STS score [2304.11434] [2211.11187]. The 2025 paper, however, does not specify label normalization; it states the use of CosineSimilarityLoss but does not detail the normalization [2508.21569].

The embedding dimensionality is documented only in the earlier work: the model inherits the base BERT hidden size, and for L3Cube “base” BERTs this is 768 dimensions [2304.11434]. By contrast, the 2025 MahaSTS paper does not report parameter count [2508.21569].

## 3. The MahaSTS dataset and its supervision regime

MahaSTS is a human-annotated Marathi Sentence Textual Similarity dataset with 16,860 sentence pairs labeled on a continuous 0–5 scale [2508.21569]. The splits are train 14,328, test 1,692, and validation 840, corresponding to an 85:10:5 ratio [2508.21569]. Its defining structural property is a uniform bucket strategy: the full dataset is uniformly distributed across six score-based buckets, each containing exactly 2,810 pairs, spanning score 0 through the interval 4.1–5.0 [2508.21569].

The bucket definitions are given explicitly as qualitative supervision guidelines. Bucket 0 corresponds to score 0 and “no semantic similarity”; bucket 1 covers 0.1–1.0 and “minimal similarity”; bucket 2 covers 1.1–2.0 and “somewhat related”; bucket 3 covers 2.1–3.0 and “moderate similarity”; bucket 4 covers 3.1–4.0 and “highly similar”; and bucket 5 covers 4.1–5.0 and “nearly or fully semantically equivalent” [2508.21569]. Per split, each bucket contains 2,388 train pairs, 282 test pairs, and 140 validation pairs [2508.21569].

The source corpus is 1 million real Marathi sentences from L3Cube-MahaCorpus [2508.21569]. Preprocessing removed sentences that were too short, defined as fewer than 3 words, too long, defined as more than 20 words, non-Marathi sentences, and duplicates [2508.21569]. For candidate selection, the team used an earlier SBERT similarity model, `l3cube-pune/marathi-sentence-similarity-sbert`, to embed the 1M sentences, randomly sampled 5,000 query sentences, and computed cosine similarity against the full corpus [2508.21569]. Candidate pairs were then retrieved by cosine-based bins, with one similar sentence per bucket per query across five nonzero similarity buckets, yielding 25,000 pairs before further filtering [2508.21569]. Completely dissimilar pairs were assigned to bucket 0, and human annotation refined the labels to the 0–5 gold scores and enforced uniform distribution across buckets; incomplete or nonsensical pairs were discarded, producing the final 16,860 pairs [2508.21569].

The dataset is Marathi in Devanagari script, and non-Marathi content was filtered out [2508.21569]. Beyond the corpus-level filtering and SBERT-based retrieval, no special tokenization or text normalization is reported for the annotations or training [2508.21569]. Annotators used prior cosine scores as a reference during annotation, but inter-annotator agreement metrics such as kappa or correlation are not reported [2508.21569].

This supervision design is notable because it differs materially from the translated STSb regime used in earlier Marathi SBERT work. The 2022 and 2023 papers trained and evaluated Marathi STS models on machine-translated STS benchmark data, preserving the original STSb split sizes of 5,749 train, 1,500 dev, and 1,379 test per language [2304.11434] [2211.11187]. MahaSTS replaces that translated-resource dependency with a human-curated, bucket-balanced Marathi dataset [2508.21569].

## 4. Training pipeline and optimization

In the 2025 formulation, MahaSBERT-STS-v2 is obtained by fine-tuning the base encoder MahaSBERT on MahaSTS for cosine-similarity regression [2508.21569]. The optimization setup is concise and explicitly reported: CosineSimilarityLoss, MEAN pooling over token embeddings, AdamW, learning rate $1e^{-5}$, batch size 8, and 2 epochs [2508.21569]. Warmup steps, max sequence length, dropout, random seed, early stopping, hardware, and training duration are not reported [2508.21569].

The broader L3Cube training lineage is two-stage. In the earlier papers, Marathi SBERT training begins with NLI pretraining on translated IndicXNLI using MultipleNegativesRankingLoss, followed by STS fine-tuning with CosineSimilarityLoss on translated Marathi STSb [2304.11434] [2211.11187]. For the NLI step, the papers report 1 epoch, batch size 4, AdamW, and learning rate $2e^{-5}$; for the STS step, they report 4 epochs with AdamW and learning rate $2e^{-5}$, with batch size 8 in the two-step STS fine-tuning stage [2304.11434] [2211.11187]. The NLI step treats entailment pairs as positives and contradiction pairs as hard negatives for the same anchor [2304.11434].

This establishes a three-level progression in the Marathi sentence-embedding line: MahaBERT as the monolingual backbone, MahaSBERT as the NLI-only sentence encoder, and MahaSBERT-STS or MahaSBERT-STS-v2 as the STS-calibrated model [2211.11187] [2508.21569]. A plausible implication is that the 2025 model inherits the semantic structuring induced by IndicXNLI-based NLI training and then calibrates it to native human similarity judgments through MahaSTS fine-tuning, but the 2025 paper specifically frames the improvement relative to the NLI-only checkpoint rather than reproducing the full earlier two-step translated-STSb pipeline [2508.21569].

## 5. Empirical performance and comparative evaluation

On the MahaSTS test set, the 2025 MahaSBERT-STS-v2 achieves Pearson correlation $r = 0.9600$ and Spearman rank correlation $\rho = 0.9523$ against human labels [2508.21569]. The same paper reports a pooling ablation on the MahaSBERT base showing MEAN $>$ CLS $>$ MAX on the test set: CLS yields Pearson 0.9580 and Spearman 0.9503; MEAN yields Pearson 0.9600 and Spearman 0.9523; MAX yields Pearson 0.9532 and Spearman 0.9444 [2508.21569].

The comparative baselines trained and evaluated on MahaSTS show the following test-set results [2508.21569]:

| Model | Pearson $r$ | Spearman $\rho$ |
|---|---:|---:|
| MahaSBERT-STS-v2 | 0.9600 | 0.9523 |
| MahaBERT | 0.9483 | 0.9386 |
| MuRIL | 0.9361 | 0.9267 |
| IndicBERT | 0.7311 | 0.7004 |
| IndicSBERT | 0.9515 | 0.9441 |

The 2025 paper further reports that fine-tuning on MahaSTS improves over the original NLI-only MahaSBERT baseline, which records Pearson 0.9355 and Spearman 0.9268 [2508.21569]. Statistical significance tests are not discussed [2508.21569].

The earlier L3Cube results provide a complementary historical baseline. On translated Marathi STSb test data, the two-step Marathi SBERT model records Spearman correlation 0.83 in both the 2022 and 2023 descriptions [2304.11434] [2211.11187]. In that setting, the model outperforms multilingual baselines such as LaBSE, MuRIL, mBERT, and IndicBERT, and improves over NLI-only and STS-only Marathi variants [2211.11187]. The earlier 2023 paper also reports that the monolingual Marathi SBERT outperforms IndicSBERT-STS on Marathi STS, 0.83 versus 0.81, while IndicSBERT-STS is stronger for cross-lingual settings [2304.11434].

These result sets are not numerically interchangeable because they are measured on different benchmarks: the older figure of 0.83 Spearman is on translated STSb, whereas the newer Pearson/Spearman values are on MahaSTS [2304.11434] [2211.11187] [2508.21569]. What they jointly support is a consistent pattern: Marathi-specific SBERT models improve markedly when trained with task-aligned similarity supervision rather than relying on generic sentence encoders alone.

## 6. Relationship to related Marathi and Indic sentence encoders

MahaSBERT-STS-v2 sits within a broader L3Cube program on Marathi and Indic sentence representations. MahaBERT is the monolingual Marathi BERT backbone; MahaSBERT is the Marathi NLI-trained sentence-BERT checkpoint; MahaSBERT-STS is the earlier STS-tuned Marathi SBERT release; IndicSBERT and IndicSBERT-STS are multilingual sentence-BERT models built on MuRIL and trained on aggregated data from ten Indic languages [2304.11434] [2211.11187].

The distinction between monolingual and multilingual objectives is methodologically important. The monolingual Marathi models use Marathi-only data and a Marathi-specific base model, which the papers position as the best choice for Marathi-only applications [2304.11434]. IndicSBERT, by contrast, is intended for a common multilingual representation space and is explicitly described as useful for cross-language similarity and mining tasks [2304.11434]. The 2023 paper reports that IndicSBERT exhibits strong cross-lingual capabilities and performs significantly better than alternatives like LaBSE, LASER, and `paraphrase-multilingual-mpnet-base-v2` on Indic cross-lingual and monolingual sentence similarity tasks [2304.11434].

For Marathi-only semantic similarity, however, the monolingual line remains stronger in the reported evaluations. The 2023 paper states that MahaSBERT-STS at 0.83 Spearman outperforms multilingual models on Marathi STS, including IndicSBERT-STS at 0.81 and LaBSE at 0.70 [2304.11434]. The 2025 MahaSTS study reaches a similar conclusion with a different benchmark and a different comparison set: MahaSBERT-STS-v2 outperforms MahaBERT, MuRIL, IndicBERT, and IndicSBERT on the MahaSTS test set [2508.21569].

This suggests a stable division of labor rather than a contradiction. MahaSBERT-STS-v2 is optimized for high-fidelity Marathi semantic similarity, whereas IndicSBERT-type models serve multilingual and cross-lingual alignment needs [2304.11434]. The evidence presented across the papers supports that interpretation without eliminating the value of multilingual encoders in mixed-language retrieval pipelines.

## 7. Practical use, limitations, and future directions

For practical inference, the model is used through Sentence-Transformers. The 2025 paper provides the model identifier `l3cube-pune/marathi-sentence-similarity-sbert-v2`, expects Marathi input in Devanagari, and states that no special preprocessing is required beyond standard transformer tokenization; training relied on MEAN pooling over token embeddings [2508.21569]. The minimal usage pattern is to encode two Marathi sentences and compute cosine similarity between their embeddings [2508.21569]. The earlier papers provide analogous examples for the earlier checkpoint `l3cube-pune/marathi-sentence-similarity-sbert` [2304.11434] [2211.11187].

The stated application profile includes sentence-level semantic similarity for retrieval, clustering, semantic search, paraphrase detection, retrieval-augmented generation, and question answering [2211.11187] [2508.21569]. The 2025 paper notes that strong sentence embeddings directly benefit downstream tasks such as paraphrase detection, semantic search, retrieval-augmented generation, clustering, and question answering [2508.21569].

Several limitations are explicit. The 2025 paper notes reduced generalization for longer or complex or compound Marathi sentences and states that SBERT-style models perform best on short to moderately long inputs [2508.21569]. The earlier work emphasizes limitations stemming from synthetic translation noise, domain mismatch, limited topical coverage, lack of bias audits, and uncertain robustness to informal, dialectal, or code-mixed Marathi, especially Romanized Marathi, which earlier preprocessing intentionally removed [2304.11434] [2211.11187]. In the MahaSTS setting, non-Marathi content is filtered out and no special normalization beyond standard tokenization is reported, so robustness outside clean Devanagari Marathi remains an open question [2508.21569].

Future directions are stated directly in the 2025 paper: develop separate datasets with sentences of varying lengths to improve generalization for complex structures, pursue more task-specific fine-tuning strategies, and continue emphasizing human-curated resources for Marathi and other Indic languages [2508.21569]. This suggests a broader research agenda in which native, bucket-balanced supervision complements or supersedes translated STS resources for low-resource languages.

Source: https://www.emergentmind.com/topics/mahasbert-sts-v2