ColmmBERT-base-TR: Turkish Neural Retriever

• The paper presents ColmmBERT-base-TR, a late-interaction neural retriever that adapts a multilingual Transformer with token-level interactions for Turkish information retrieval.
• The methodology utilizes a two-stage adaptation process—semantic pre-adaptation followed by ColBERT-style retrieval—and advanced indexing algorithms like PLAID and MUVERA for balancing speed and accuracy.
• Empirical results on Turkish benchmarks reveal significant mAP improvements and enhanced parameter efficiency, suggesting strong potential for scalable, low-latency deployments in morphologically rich settings.

ColmmBERT-base-TR is a late-interaction neural retriever model for Turkish information retrieval, combining semantic adaptation of a multilingual Transformer backbone with ColBERT-style token-based retrieval and efficient approximate indexing. Designed and evaluated within the TurkColBERT benchmark, the model supports fine-grained token interactions, parameter-efficient retrieval, and high-throughput indexing for low-latency deployment in morphologically rich, lower-resource Turkish settings (Ezerceli et al., 20 Nov 2025).

1. Model Architecture and Adaptation

ColmmBERT-base-TR leverages a 12-layer multilingual Transformer backbone (“mmBERT-base”) with hidden size $d=768$, 12 attention heads, and approximately 110 million parameters in its base pretrained configuration. Through conversion with PyLate to a ColBERT-style late-interaction retriever, the model—augmented by projection layers and token-wise modules—reaches approximately 310 million parameters.

Adaptation to Turkish proceeds via a two-stage pipeline:

• Stage 1: Monolingual Semantic Pre-adaptation leverages Turkish natural language inference (NLI) and semantic textual similarity (STS) tasks:
  • NLI: Turkish SNLI & MultiNLI translations (~433k examples), Siamese network, MultipleNegativesRankingLoss wrapped in MatryoshkaLoss across embedding sizes $[768,512,384,256,128,64]$ (1 epoch, BS=8, LR=$3\times10^{-6}$, BF16).
  • STS: STSb-TR (~8k sentence pairs), cosine regression loss, 4 epochs, BS=8, LR=$2\times10^{-5}$, cosine schedule.
  • Performance: Spearman = 0.78 (STS), triplet accuracy = 93% (NLI).
• Stage 2: ColBERT-Style Retrieval Adaptation employs contrastive triplets from MS MARCO-TR (~532k queries × 8.8M passages) for per-token embedding preservation and late-interaction retrieval. Training uses PyLate defaults (BS ≈ 32–64, LR = $3\times10^{-5}$) and early stopping on validation MRR.

Token-level retrieval is realized with ColBERT MaxSim: $$\text{score}(Q, D) = \sum_{i=1}^{|Q|} \max_{j=1 \dots |D|} \langle e(q_i), e(d_j) \rangle$$ where $e(\cdot)$ denotes $\ell_2$-normalized token embeddings.

2. Indexing, Hashing, and Acceleration

ColmmBERT-base-TR enables high-throughput deployment via two major indexing algorithms:

• PLAID (exact): Employs centroid-based pruning and residual compression to select and compress candidate document sets, permitting exact MaxSim computation over a reduced pool.
• MUVERA: An approximate, fixed-dimension indexing approach using SimHash-based partitioning and AMS-sketch-style aggregation. Each token embedding is assigned to one of $2^k$ partitions (via $k$ random Gaussian vectors and the sign function), and per-partition aggregated representations $c_p$ are concatenated for fixed-dimensional document encoding. On retrieval, approximate nearest neighbor (ANN) techniques are used for fast search.
• MUVERA + Rerank: Retrieves the top-$K$ candidates by MUVERA, followed by full ColBERT MaxSim rescoring for quality-latency tradeoff.

Indexing performance and latency are critical: PLAID enables exact search at higher cost (e.g., on SciFact-TR, query latency ≈ 73.6 ms, mAP ≈ 0.3257), while MUVERA achieves sub-millisecond latency (0.54–0.72 ms, slight mAP reduction), and MUVERA+Rerank recovers maximal performance with intermediate latency (35.2 ms, +1.7% mAP over PLAID using $K=100$).

3. Parameter Efficiency and Ablation

ColmmBERT-base-TR demonstrates considerable parameter efficiency compared to dense baseline retrievers:

Model	Params (M)	Avg. mAP	mAP Retention vs. 600M
turkish-e5-large	600	16.4%	–
ColmmBERT-base-TR	310	22.4%	–
ColmmBERT-small-TR	140	20.0%	≈97.5%
colbert-hash-nano-tr	1.0	12.7%	77.6%
colbert-hash-femto-tr	0.2	6.7%	40.8%

ColBERT-Hash “nano” and “femto” variants—parameter-reduced models that replace standard token embeddings with hash-based projections and compact lightweight matrices—offer further reductions in memory and computation, enabling deployment on edge devices. The “nano” model achieves >78% parameter reduction versus standard embeddings while retaining full vocabulary coverage.

4. Empirical Performance on Turkish IR Benchmarks

Evaluation on five Turkish BEIR datasets (SciFact-TR, Arguana-TR, Fiqa-TR, Scidocs-TR, NFCorpus-TR) demonstrates that ColmmBERT-base-TR consistently outperforms strong dense baselines. Absolute mAP improvements reach +13.8 points on SciFact-TR relative to TurkEmbed4Retrieval, with an average gain of +36.6% against dense models such as turkish-e5-large.

Dataset	TurkEmbed4Ret	turkish-e5-large	ColmmBERT-base-TR
SciFact-TR	43.0%	45.8%	56.8% (+13.8)
Arguana-TR	17.6%	17.9%	17.3% (–0.3)
Fiqa-TR	10.1%	10.4%	19.5% (+9.1)
Scidocs-TR	4.8%	2.2%	6.8% (+2.0)
NFCorpus-TR	6.3%	4.0%	11.5% (+5.2)

Significant gains are observed particularly in scientific and financial retrieval tasks. A plausible implication is that late-interaction modeling with token-level representations is particularly effective in morphologically rich Turkish domains.

5. Production Deployment and Latency

ColmmBERT-base-TR, indexed with MUVERA and deployed under production configurations (512D index), achieves query times as low as 0.54 ms—suitable for interactive Turkish search systems. MUVERA indexing is 3.3× faster than PLAID while delivering competitive mAP (≤1.7% reduction, offset by reranking if desired). The model and its parameter-efficient variants provide modularity for deployment scenarios ranging from cloud backends to resource-constrained edge devices.

Deployment considerations include the requirement for a pre-computed FDE index, with memory scaling linearly in corpus size and vector dimension. For applications prioritizing both latency and ranking quality, MUVERA+Rerank presents an optimal point on the trade-off curve.

6. Limitations and Open Issues

Evaluation in TurkColBERT is restricted to moderately sized Turkish corpora (≤50,000 documents), with all BEIR tasks translated from English. As such, some domain idiosyncrasies of native Turkish corpora may not be fully captured, and generalization to larger-scale retrieval tasks (>100k documents) remains untested. Further large-scale benchmarking of MUVERA indexing on native Turkish text is necessary to validate real-world applicability. A plausible implication is that future research should address both data scale and original corpus characteristics to comprehensively assess deployment readiness (Ezerceli et al., 20 Nov 2025).