ViRanker: Vietnamese Reranking Model

• ViRanker is a Vietnamese cross-encoder reranking model tailored for low-resource language challenges with complex syntax and diacritics.
• It integrates Rotary Position Encoding and Blockwise Parallel Transformer to boost long-context efficiency and diacritic sensitivity.
• Evaluated on MMARCO-VI, ViRanker achieves superior early-rank accuracy compared to both multilingual baselines and PhoRanker.

Searching arXiv for the ViRanker paper and closely related items to ground the article in current arXiv records. arxiv_search(query="ViRanker BGE-M3 Blockwise Parallel Transformer Vietnamese reranking", max_results=5) ViRanker is a cross-encoder reranking model tailored to the Vietnamese language and presented in "ViRanker: A BGE-M3 & Blockwise Parallel Transformer Cross-Encoder for Vietnamese Reranking" (Dang et al., 11 Sep 2025). It is built on the BGE-M3 encoder, augmented with Rotary Position Encoding (RoPE) and the Blockwise Parallel Transformer (BPT), and uses a lightweight MLP head to produce a scalar relevance score. The model is positioned as a response to the lack of competitive rerankers for Vietnamese, described as a low-resource language with complex syntax and diacritics, and was trained on an 8 GB curated corpus before evaluation on the MMARCO-VI benchmark, where it achieved strong early-rank accuracy and competed closely with PhoRanker (Dang et al., 11 Sep 2025).

1. Definition and retrieval setting

ViRanker operates in the reranking stage of a retrieval pipeline. In the reported evaluation setting, MMARCO-VI mimics MS-MARCO but in Vietnamese, offering approximately 6,000 test queries and approximately 100 candidates per query for reranking; the primary task is to reorder a BM25-retrieved set by relevance (Dang et al., 11 Sep 2025).

Within that setting, ViRanker is explicitly a cross-encoder reranking model rather than a first-stage retriever. Query and document are processed jointly, and the model outputs a scalar relevance score $s(q,d)$ used for ranking. This design places it in the class of neural rerankers that trade higher scoring cost for improved ranking precision, especially at small cut-offs. The reported emphasis on early-rank metrics, particularly $k=3$, indicates that the model is optimized for scenarios in which the quality of the top few retrieved items is operationally important.

The paper frames Vietnamese reranking as a technically distinctive problem because of flexible word order and the subtleties of diacritics. This motivates architectural modifications to a multilingual pretrained encoder rather than relying on an unmodified multilingual baseline.

2. Architecture

ViRanker is built on top of the BGE-M3 embedding encoder and enhanced with two Vietnamese-specific adaptations: RoPE and BPT. A lightweight MLP head on top converts the encoder’s final representations into a scalar relevance score (Dang et al., 11 Sep 2025).

The backbone weights of BGE-M3 are retained. To better capture Vietnamese’s flexible word order and the subtleties of diacritics, the original absolute positional embeddings are replaced with Rotary Position Encoding. In the formulation given for RoPE, each pair of hidden dimensions splits into a 2D rotation by angle $\theta$ proportional to token position $p$:

$$\mathrm{RoPE}(x, p) = R(p)\,x, \quad R(p) = \begin{pmatrix} \cos\theta_p & -\sin\theta_p\ \sin\theta_p & \cos\theta_p \end{pmatrix}, \quad \theta_p = p / 10000^{2i/d}\,.$$

The paper states that this preserves relative phase differences and yields stronger modeling of Vietnamese word-order phenomena.

The second major architectural element is the Blockwise Parallel Transformer. Standard self-attention on a sequence of length $L$ requires $O(L^2)$ memory and compute. BPT alleviates this by splitting the sequence into $B$ blocks and interleaving local feed-forward computation with blockwise attention. The sequence is partitioned into blocks $X_i$, a local feed-forward update is computed in parallel,

$\widetilde X_i = \mathrm{FFN}(X_i),$

and cross-block attention is then applied as

$k=3$0

These steps are interleaved for $k=3$1 layers to approximate full self-attention with dramatically lower per-layer memory.

After the last BPT layer, ViRanker applies mean-pooling over the concatenated query+document hidden states, producing a single vector $k=3$2. A two-layer MLP computes the final relevance score:

$k=3$3

This architecture combines a retained multilingual pretrained backbone with positional and attention modifications intended to improve Vietnamese-specific modeling and long-context efficiency.

3. Corpus construction and preprocessing

To obtain a large, diverse Vietnamese corpus, the study collected 8 GB of raw text from three major sources: Vietnamese Wikipedia, open-source GitHub repositories with Vietnamese readme/doc files, and books and reports in Vietnamese (Dang et al., 11 Sep 2025). The corpus is described in the paper as follows.

Source	Size	Description
Vietnamese Wikipedia	2.1 GB	$k=3$4 M documents; general-knowledge articles
GitHub repositories with Vietnamese readme/doc files	1.4 GB	$k=3$5 M docs; technical text
Books and reports in Vietnamese	4.5 GB	$k=3$6 M docs; long-form formal writing

Preprocessing consisted of punctuation and diacritic normalization, including the example of unifying “hoà” to “hòa”; automated spelling correction via Gemma-1.5 and GPT-4o-mini; and sentence segmentation with discarding of fragments under 512 tokens and merging of adjacent segments to produce chunks up to 1024 tokens. This yielded 3.5 million document chunks with average approximately 730 tokens per chunk.

From these chunks, the paper generated 3.5 million training triplets via an Inverse Cloze Task. For each chunk, one sentence serves as a pseudo-query $k=3$7, the remainder serves as the positive passage $k=3$8, and three hard negatives $k=3$9 are mined in four stages: BM25 retrieves the top 20 candidate passages, these candidates are reranked by cosine similarity of BGE-M3 embeddings, Maximal Marginal Relevance is applied to diversify them, and the three most challenging non-relevant passages are selected.

A plausible implication is that the training pipeline was designed to compensate for the relative scarcity of supervised Vietnamese reranking data by constructing large-scale pseudo-supervision while preserving semantic difficulty through hybrid hard-negative mining.

4. Training objective and optimization

ViRanker is trained with the standard triplet-ranking, or contrastive, loss (Dang et al., 11 Sep 2025):

$\theta$0

where the preset margin $\theta$1 is set to $\theta$2. Minimizing this objective pushes the positive score above every negative by at least the margin.

The hard-negative sampling strategy is explicitly hybrid: BM25 retrieval, dense BGE-M3 reranking, and MMR diversification are applied on each pseudo-query. The paper states that this forces the model to discriminate between semantically close but non-relevant passages, greatly improving robustness in fine-tuning.

The key training settings reported in Table 3 are a maximum sequence length of 1024 tokens, train/eval batch size of 512/512 triplets, 32 attention heads, a learning rate of $\theta$3 with cosine decay scheduler, 4 epochs, gradient checkpointing and accumulation, mean-pooling with a 2-layer MLP scoring head, and the AdamW optimizer. The training/validation split within the 3.5 million triplets was held out at random for early stopping at approximately 5–10%, while the unseen MMARCO-VI test set measures final performance.

These settings indicate that the model was trained under long-context conditions rather than being restricted to short passages, and that memory-management techniques were necessary to support large batches at 1024-token sequence length.

5. Evaluation and reported performance

The study evaluates ViRanker on the official MMARCO-VI benchmark. The reported metrics are Mean Reciprocal Rank and Normalized Discounted Cumulative Gain at $\theta$4 (Dang et al., 11 Sep 2025). The formulas given are

$\theta$5

and

$\theta$6

The main quantitative comparison reported in the paper is the following.

Metric	ViRanker	PhoRanker
NDCG@3	0.6815	0.6625
NDCG@5	0.6983	0.7147
NDCG@10	0.7302	0.7422
MRR@3	0.6641	0.6458
MRR@5	0.6894	0.6731
MRR@10	0.7107	0.6830

The paper also reports that multilingual BGE variants achieve approximately $\theta$7–$\theta$8 on NDCG@3 and approximately $\theta$9–$p$0 on MRR@3. On that basis, the study concludes that ViRanker delivers the strongest early-rank accuracy, especially at $p$1, outperforming both multilingual baselines and the previous Vietnamese state-of-the-art PhoRanker at those cut-offs. At $p$2 and $p$3, the reported comparison is more mixed: PhoRanker is higher on NDCG@5 and NDCG@10, whereas ViRanker remains higher on MRR@5 and MRR@10.

This performance profile suggests that the model’s advantages are concentrated in the earliest ranking positions rather than uniformly across all cut-offs.

6. Strengths, limitations, reproducibility, and disambiguation

The paper identifies three principal strengths of ViRanker: early-rank gains, long-document efficiency, and robustness (Dang et al., 11 Sep 2025). Early-rank gains are attributed to RoPE’s diacritic- and word-order sensitivity; long-document efficiency is attributed to BPT enabling 1024-token contexts at scale; and robustness is attributed to hybrid hard-negative mining that forces discrimination of near-duplicates.

The reported limitations are also explicit. Short factoid queries such as “Nguyễn Du năm sinh” can still be mis-ranked under named-entity ambiguity. Very long narrative documents may dilute relevant context. The 8 GB corpus, while carefully curated, remains small compared to high-resource languages and may not cover niche domains. Sensitivity to input noise, including misspellings and stripped diacritics, requires further analysis. These constraints are important because they delimit the conditions under which the observed gains should be interpreted.

On reproducibility and deployment, the model is publicly released on Hugging Face as namdp-ptit/ViRanker, together with training scripts and dataset cards for MMARCO-VI. Inference efficiency was tested on A100, V100, and T4 GPUs, showing sub-40 ms/query latency even on T4 at batch size $p$4. The paper further states that the combination of a strong multilingual or monolingual encoder, language-tailored positional encodings, scalable long-context attention, and rigorous hybrid negative mining forms a blueprint for building high-accuracy rerankers in other under-represented languages.

A common source of confusion is nominal similarity with unrelated systems. ViRanker should be distinguished from VRank, which is an automatic framework for enhancing Verilog code generation from LLMs via self-consistency (Zhao et al., 22 Jan 2025), and from TRIVEA, which is a visual analytic system for transparent ranking interpretation using surrogate learning-to-rank models with LIME and ICE explanations (Yuan et al., 2023). The three systems address different technical problems: Vietnamese neural reranking, Verilog code generation, and ranking interpretability, respectively.