TurkColBERT: Efficient Turkish IR Models

• The paper introduces TurkColBERT, a novel framework that benchmarks ColBERT-style late-interaction models for Turkish IR, achieving high triplet accuracy (~93%) and strong retrieval metrics.
• TurkColBERT employs a two-stage adaptation pipeline—semantic pre-adaptation followed by late-interaction via PyLate—to finely tune models for Turkish morphological complexity and efficient query processing.
• The approach outperforms dense bi-encoders on benchmarks like FiQA-TR and Scidocs-TR by delivering superior retrieval effectiveness and parameter efficiency, with query latencies as low as 0.54 ms.

TurkColBERT denotes a family of information retrieval (IR) models, methodologies, and benchmarks that systematically explore the application of late-interaction neural retrieval architectures—particularly ColBERT-style models—for the Turkish language. Emerging from a context where dense bi-encoders have dominated Turkish IR, TurkColBERT represents the first comprehensive effort to benchmark dense and late-interaction models for Turkish across scientific, financial, and argumentative domains. These models are tailored to the morphological complexity and agglutinative structure of Turkish, achieving high retrieval effectiveness and remarkable parameter efficiency relative to dense encoders (Ezerceli et al., 20 Nov 2025).

1. Methodological Foundations and Adaptation Pipeline

TurkColBERT adopts a two-stage model adaptation pipeline designed for information retrieval in Turkish:

Stage 1: Semantic Pre-Adaptation

Base multilingual models—including mmBERT-base (110 M parameters), mmBERT-small (45 M), Ettin (150 M/32 M), and BERT-Hash variants (nano, pico, femto)—are fine-tuned on Turkish natural language inference (All-NLI-TR) and semantic textual similarity (STS-B-TR) datasets. Training uses Sentence-Transformers’ MultipleNegativesRankingLoss wrapped in a MatryoshkaLoss to facilitate multi-dimensional outputs spanning the final to lower-dimensional spaces. Pre-adaptation produces sentence or token representations with high triplet accuracy (~93%) and Spearman correlation (~0.78) on validation splits (Ezerceli et al., 20 Nov 2025).

Stage 2: Late-Interaction Adaptation

Pre-adapted checkpoints are converted into ColBERT-style retrievers using the PyLate framework. For each input, token embeddings are projected to $d=128$ dimensions and retained in their entirety. Scoring employs MaxSim, where a query and a document receive a similarity score computed as:

$$\text{score}(q, d) = \sum_{i=1}^{|q|} \max_{1 \le j \le |d|} \langle \mathbf{q}_i, \mathbf{d}_j \rangle$$

A margin-based triplet loss (margin 0.2) supervises training on MS MARCO-TR passage triplets (Ezerceli et al., 20 Nov 2025).

Stage 3: Indexing with MUVERA

Indexing incorporates the MUVERA Fixed Dimensional Encoding (FDE) algorithm, featuring SimHash partitioning, AMS sketching, and partition-wise aggregation for scalable approximate nearest neighbor (ANN) search. This enables sub-millisecond per-query latency, facilitating production deployment. Models with 1–310 million parameters can achieve as little as 0.54 ms/query (Ezerceli et al., 20 Nov 2025).

2. Model Architectures and Parameter Efficiency

TurkColBERT benchmarks both dense bi-encoder and ColBERT-style late-interaction models:

• Dense Bi-Encoders: TurkEmbed4Retrieval (300 M parameters; mean-pooled BERT-base architecture), turkish-e5-large (600 M; E5-large).
• Late-Interaction Models: turkish-colbert (100 M; standard ColBERTv1), ColmmBERT-small-TR (140 M), ColmmBERT-base-TR (310 M), col-ettin variants, and BERT-Hash compressions down to 0.2 M parameters.

Late-interaction models demonstrate substantial parameter efficiency. For example, colbert-hash-nano-TR, with just 1 million parameters, achieves over 71% of the average mAP of the 600 million parameter turkish-e5-large dense encoder ((36.2+10.5+6.5+3.6+6.7)/5 vs (45.8+17.9+10.4+2.2+4.0)/5) (Ezerceli et al., 20 Nov 2025).

Model	Parameters (M)	Architecture
TurkEmbed4Retrieval	300	Dense, mean-pooled
turkish-colbert	100	ColBERT late-interaction
ColmmBERT-base-TR	310	ColBERT late-interaction
colbert-hash-nano-TR	1.0	Token-hash, SimHash 128D

3. Evaluation: Retrieval Effectiveness and Latency

Evaluation leverages the BEIR framework on five Turkish benchmarks: SciFact-TR, Arguana-TR, FiQA-TR, Scidocs-TR, and NFCorpus-TR. Metrics include Precision@K, Recall@K, and mean Average Precision (mAP):

Model	SciFact	Arguana	FiQA	Scidocs	NFCorpus
TurkEmbed4Retrieval	43.0	17.6	10.1	4.8	6.3
turkish-colbert	43.1	14.6	11.3	2.8	6.9
ColmmBERT-base-TR	56.8	17.3	19.5	6.8	11.5
colbert-hash-nano-TR	36.2	10.5	6.5	3.6	6.7

The best late-interaction model (ColmmBERT-base-TR) outperforms the best dense encoder by +13.8 mAP absolute on FiQA-TR (19.5 vs 10.4) and +209% relative on Scidocs-TR (Ezerceli et al., 20 Nov 2025).

Latency studies reveal that MUVERA+Rerank achieves query times of 27–35 ms for top-late-interaction models, with ColmmBERT-base-TR under MUVERA reaching 0.54 ms per query. Parameter-reduced models such as ColmmBERT-small-TR maintain 97.5% of base performance at ~45% of the computational cost (Ezerceli et al., 20 Nov 2025).

4. Comparative Analysis with Dense Retriever and Hybrid Extensions

Dense bi-encoders exhibit strong absolute performance but are outperformed in both parameter efficiency and retrieval effectiveness by late-interaction models. Specifically, multi-granularity ColBERT architectures and two-stage pipelines—where an initial dense encoder such as TurkEmbed4Retrieval provides ANN candidates and a ColBERT model re-ranks—are cited as promising extensions.

Key architectural variants outlined for future TurkColBERT development include:

• Cached Multiple Negatives Ranking Loss: Integration of cached negatives into ColBERT training in place of standard in-batch negatives to improve negative sampling diversity.
• Matryoshka Token Embeddings: Combining representations from multiple Transformer depths (instead of only the last layer) to better accommodate Turkish morphology (Ezerceli et al., 10 Nov 2025).
• Cross-Objective Multitask Learning: Simultaneous global-embedding and token-level ColBERT losses for robust retrieval (Ezerceli et al., 10 Nov 2025).

5. Indexing Strategies and Scalability

TurkColBERT includes extensive analysis of ANN indexing for token-level representations. The MUVERA algorithm performs SimHash partitioning, AMS sketching, and partition-wise aggregation, supporting dimensions $D = d \times 2^k$ for $k = 0,2,3,4$. This enables efficient search across datasets of up to 50K documents.

Comparative results demonstrate:

• PLAID: High recall, but slow query times (73–124 ms/query).
• MUVERA: 0.72–1.5 ms/query (128–512D), with 5–10% drop in NDCG.
• MUVERA+Rerank: 27–35 ms/query, recovering >95% of PLAID’s quality, with +1.7% relative mAP over plain MUVERA (Ezerceli et al., 20 Nov 2025).

6. Limitations and Prospects for Turkish IR

Limitations of TurkColBERT include reliance on moderately-sized and translated datasets ($\leq$50K documents), which may not fully reflect the scale or linguistic diversity of real-world Turkish retrieval. The benchmark currently lacks evaluation on web-scale document collections ($\geq10^6$). Morphological and inflectional specifics suggest that dedicated tokenization and hybrid sparse-dense modeling should be explored. Planned directions involve the development of native Turkish retrieval corpora, morphology-aware late interaction models, hybrid dense-late interaction pipelines, and integration of generative rerankers (Ezerceli et al., 20 Nov 2025).

A plausible implication is that further integration of Matryoshka representation learning and large-scale hard-negative mining can yield further gains for Turkish ColBERT-style models (Ezerceli et al., 10 Nov 2025).

7. Significance and Public Resources

TurkColBERT establishes that late-interaction neural retrieval models—notably those adapted via a two-stage pipeline and efficient ANN indexing—can outperform much larger dense encoders for Turkish across multiple domains, while achieving extreme parameter and latency efficiency. All code, checkpoints, and evaluation scripts have been publicly released, rendering TurkColBERT a reference platform for future Turkish IR research (Ezerceli et al., 20 Nov 2025).