NanoVDR: Asymmetric Visual Document Retrieval

• NanoVDR is an asymmetric visual document retrieval architecture that uses a frozen 2B vision-language model for offline document indexing and a compact 70M text-only encoder for queries.
• The system employs direct cosine alignment for distillation, outperforming ranking-based losses and significantly reducing training and online computational costs.
• NanoVDR achieves 95.1% of teacher quality with 32× fewer parameters and 50× faster CPU query latency, making it ideal for large-scale enterprise visual document search.

Searching arXiv for NanoVDR and closely related work to ground the article. Calling arXiv search for "NanoVDR". NanoVDR is an asymmetric visual document retrieval (VDR) architecture that distills a large vision-language retriever into a compact text-only query encoder while preserving the original document-side visual representation. In the formulation reported in “NanoVDR: Distilling a 2B Vision-Language Retriever into a 70M Text-Only Encoder for Visual Document Retrieval” (Liu et al., 13 Mar 2026), a frozen 2B vision-LLM indexes document pages offline, and a distilled student with as few as 69M parameters encodes text queries at inference time. The design is motivated by query–document asymmetry: document pages are visually complex, whereas queries are short text strings. The resulting system removes the large multimodal encoder from the online path, retains 95.1% of teacher quality in its best variant, and reduces CPU query latency by 50× relative to a 2B baseline while using 32× fewer parameters (Liu et al., 13 Mar 2026).

1. Definition and problem setting

NanoVDR operates in the setting of visual document retrieval, where the corpus consists of document-page images and the input is a text query. The task is to rank pages by relevance without reducing the documents to OCR text alone. The paper defines the document set as

$\mathcal{D} = \{d_1, \ldots, d_N\},$

with a text query $q$, and retrieval is performed in a shared embedding space (Liu et al., 13 Mar 2026).

The architecture is explicitly designed against the dominant symmetric VLM retriever pattern, in which the same multimodal model encodes both pages and queries. NanoVDR argues that this symmetry is unnecessary for VDR because only the document side requires visual understanding. The query side can be replaced by a compact text-only encoder trained to land in the teacher’s embedding space (Liu et al., 13 Mar 2026).

This asymmetry distinguishes NanoVDR from other compact “nano” systems in the provided literature. “Fast Sparse 3D Convolution Network with VDB” uses NanoVDB as a sparse tensor storage format for 3D CNN inference rather than retrieval (Zhou et al., 2023). “NanoVer” is an XR framework for interactive molecular dynamics (Wonnacott et al., 26 Jun 2026), and “NanoVLA” concerns edge-efficient robotic policies (Chen et al., 29 Oct 2025). The term NanoVDR therefore refers specifically to visual document retrieval rather than volumetric data structures, XR, or robotics.

2. Architectural formulation

NanoVDR is a decoupled retriever with a frozen teacher for document indexing and a distilled student for query encoding. The teacher $g$ encodes document images offline as

$\mathbf{v}^D_j = g(d_j) \in \mathbb{R}^d,$

and the student $f_\theta$ encodes the query at inference time as

$\mathbf{v}^Q_s = f_\theta(q) \in \mathbb{R}^d.$

Retrieval then uses

$$\text{score}(q, d_j) = {\mathbf{v}^Q_s}^\top \mathbf{v}^D_j.$$

Because the embeddings are normalized, this is cosine similarity in practice (Liu et al., 13 Mar 2026).

The student follows a Sentence-BERT-style design: $$f_\theta(q) = \text{norm}\!\big(\text{MLP}(\text{pool}(h(q)))\big),$$ with projection head

$$\text{MLP}(\mathbf{x}) = W_2 \, \sigma(W_1 \mathbf{x} + b_1) + b_2.$$

The paper uses mean pooling over token outputs, a two-layer MLP, and final normalization (Liu et al., 13 Mar 2026).

The teacher is Qwen3-VL-Embedding-2B with embedding dimension 2048. Three student backbones are trained:

Variant	Backbone	Parameters
NanoVDR-S	DistilBERT	66M + 2M projector = 69M
NanoVDR-M	BERT-base	110M + 2M projector = 112M
NanoVDR-L	ModernBERT-base	149M + 2M projector = 151M

All students use the same $768 \to 768 \to 2048$ projector (Liu et al., 13 Mar 2026).

A central systems property follows from this design. The teacher is used only for offline document indexing in the deployed system, whereas online inference requires only the small text encoder and nearest-neighbor search over precomputed document vectors. This suggests NanoVDR should be understood not merely as model compression, but as pathway decoupling: the document pathway remains multimodal and heavy, whereas the query pathway is distilled, text-only, and CPU-feasible.

3. Distillation objective and training methodology

The defining technical result of NanoVDR is that direct pointwise cosine alignment to teacher query embeddings outperforms ranking-based distillation. The teacher also encodes training queries in text-only mode,

$q$0

and the main objective is

$q$1

This objective requires only teacher query embeddings and does not require document images, document embeddings, or negative sampling during student training (Liu et al., 13 Mar 2026).

The paper also evaluates a combined loss

$q$2

where ranking distillation uses KL divergence over in-batch similarity distributions,

$q$3

$q$4

$q$5

with $q$6 and $q$7. An InfoNCE baseline is also tested: $q$8 Across three backbones and 22 ViDoRe datasets, pure alignment is best (Liu et al., 13 Mar 2026).

The loss ablation reported in the paper is monotonic with respect to alignment weight:

$q$9	$g$0	v1	v2	v3
1	0	82.2	61.4	44.1
1	0.5	81.6	59.8	42.8
1	1	81.5	59.1	42.5
0.5	1	81.5	58.6	42.1
0	1	81.1	57.4	41.6
InfoNCE	hard	71.5	39.8	30.0

The paper interprets this as evidence that the teacher embedding space is already geometrically meaningful, and that coordinate-level imitation preserves more useful information than ranking-only supervision (Liu et al., 13 Mar 2026).

A common misconception would be to assume that retrieval distillation should primarily use retrieval-style ranking losses. NanoVDR reports the opposite. In this setting, pointwise embedding alignment is both more accurate and cheaper to train because the best objective needs only cached teacher query vectors (Liu et al., 13 Mar 2026).

4. Data, benchmarks, and training pipeline

NanoVDR is trained from 726K query-document image pairs after filtering and deduplication, split into 711,603 training pairs and 14.5K validation pairs. The sources are VisRAG-Synthetic, ColPali training set, VisRAG-InDomain, and VDR-Multilingual in Spanish, German, French, Italian, and English (Liu et al., 13 Mar 2026).

For the best align-only setting, the document side of each pair becomes irrelevant after pairing. Training consumes only query text and cached teacher query embeddings. Teacher query pre-caching costs 0.8 GPU-hours, whereas ranking losses requiring teacher document embeddings would incur 24 GPU-hours of teacher image encoding (Liu et al., 13 Mar 2026).

The evaluation benchmark is the full public ViDoRe benchmark with 22 datasets across three versions. ViDoRe v1 contains 10 datasets, v2 contains 4 multilingual datasets, and v3 contains 8 datasets spanning finance, HR, energy, industrial, pharma, physics, and computer science. The primary metric is NDCG@5 (Liu et al., 13 Mar 2026).

Training uses NVIDIA H200 GPUs with OneCycleLR, peak learning rate $g$1, 3% warmup, batch size 256, gradient accumulation 4, effective batch size 1024, 20 epochs, and about 13.9K total steps. Training times are 10.1 h for NanoVDR-S, 10.5 h for NanoVDR-M, and 11.7 h for NanoVDR-L. For the best model, including teacher query pre-caching, total training cost is under 13 GPU-hours (Liu et al., 13 Mar 2026).

This training profile is unusually lightweight for a retriever built from a 2B teacher. A plausible implication is that NanoVDR’s main innovation is not only in runtime efficiency but also in the elimination of document-side computation during distillation.

5. Multilingual transfer and the role of augmentation

The paper identifies cross-lingual transfer as the main bottleneck rather than the modality gap. Language-wise retention for NanoVDR-S is reported as 94.3% for English, 92.1% for French, 90.0% for Italian, 89.7% for Spanish, 85.7% for German, and 75.6% for Portuguese (Liu et al., 13 Mar 2026).

To address this, the authors machine-translate about 489K English queries into Portuguese, Spanish, German, French, and Italian, targeting about 200K queries per language and producing 777,649 translated queries. Combined with the original 711,603 examples, the multilingual training set reaches 1,489,252 pairs. These translated queries inherit the original positive document pairing, but align-only training still requires only the translated query and its teacher-generated target embedding (Liu et al., 13 Mar 2026).

The multilingual variant NanoVDR-S-Multi improves benchmark scores from 60.5 to 61.9 on ViDoRe v2 and from 43.5 to 46.5 on v3, while v1 remains 82.2. Per-language gains are +1.5 for French, +1.6 for Italian, +1.8 for Spanish, +3.1 for German, and +9.3 for Portuguese. After augmentation, all six languages exceed 92% retention, and the maximum cross-lingual retention gap shrinks from 18.6 percentage points to 2.7 percentage points (Liu et al., 13 Mar 2026).

This suggests that the decisive limitation in query-side distillation is not that the student never sees images. Rather, the decisive limitation is language coverage in the teacher-aligned query space. The paper explicitly supports that interpretation by noting that English retention remains high across 20 visually diverse datasets (Liu et al., 13 Mar 2026).

6. Empirical performance and deployment profile

NanoVDR’s main results are reported against the teacher and several strong baselines. The teacher Qwen3-VL-Embedding-2B scores 84.3 on v1, 65.3 on v2, and 50.0 on v3. NanoVDR-S scores 82.2, 60.5, and 43.5. NanoVDR-S-Multi scores 82.2, 61.9, and 46.5, retaining 95.1% of teacher quality overall (Liu et al., 13 Mar 2026).

Compared with DSE-Qwen2 (2B), which scores 85.1, 55.7, and 41.3, NanoVDR-S-Multi trails on v1 but surpasses it on the harder v2 and v3 benchmarks. It also surpasses ColPali (3B) on v2 and v3 (Liu et al., 13 Mar 2026).

The deployment profile is summarized below.

System	Params	CPU query latency	Checkpoint size
DSE-Qwen2	2.209B	2,539 ms	8.8 GB
NanoVDR-S	69M	51 ms	274 MB
NanoVDR-M	112M	101 ms	—
NanoVDR-L	151M	109 ms	—

The paper states that NanoVDR-S is 50× faster than DSE-Qwen2 on CPU query encoding, uses about 32× fewer parameters, and achieves under 13 GPU-hours total training cost for the best model (Liu et al., 13 Mar 2026).

NanoVDR also inherits the teacher’s single-vector retrieval format. Document tokens per page remain 1, and index size per 1M documents is 8.2 GB, compared with 264 GB for ColPali, 256 GB for ColNomic-7B, and 819 GB for Tomoro-8B. Scoring latency against 10K candidates is 2.5 ms for NanoVDR-S, close to DSE-Qwen2 at 2.3 ms, and vastly below multi-vector models (Liu et al., 13 Mar 2026).

A common misconception would be to read NanoVDR as a state-of-the-art absolute-quality retriever. The paper does not make that claim. It explicitly reports that NanoVDR still lags its own teacher and the strongest large multi-vector VLMs such as Tomoro-8B and ColNomic-7B, especially on v3 (Liu et al., 13 Mar 2026). Its contribution is the quality–efficiency tradeoff, not absolute frontier performance.

7. Significance, limitations, and relation to adjacent research

NanoVDR is significant because it shows that VDR deployment need not preserve the symmetric multimodal encoding pipeline that dominates current retrievers. The large VLM is retained only where vision is required: offline document indexing. Query encoding becomes a small text-only problem. This suggests a broader design principle for asymmetric retrieval systems in which one side of the matching function is intrinsically more complex than the other (Liu et al., 13 Mar 2026).

The paper is explicit about its limits. Student quality is capped by teacher quality because the document space is inherited from the frozen teacher. Offline indexing cost is unchanged, since all document images must still be encoded by the full 2B VLM. Evaluation is limited to first-stage visual document retrieval with text-only queries. The paper also notes that machine-translated queries may introduce semantic drift or struggle with domain-specific terminology (Liu et al., 13 Mar 2026).

In the broader provided corpus, NanoVDR sits within a family of “nano” systems that compress or decouple expensive pipelines while preserving high-value representations. “NanoVLA” similarly decouples vision and language processing for robotic policies on edge devices (Chen et al., 29 Oct 2025). “Fast Sparse 3D Convolution Network with VDB” reports a sparse 3D inference implementation using NanoVDB to reduce memory footprint and accelerate high-resolution classification (Zhou et al., 2023). These analogies are only architectural. NanoVDR itself is a retrieval system, not a robotics policy, sparse voxel network, or XR platform.

For enterprise search over PDFs, scientific papers, manuals, or reports, NanoVDR provides a particularly clean serving model: document images are indexed once on GPU, query encoding runs on CPU in about 50 ms, and scoring remains single-vector and compact (Liu et al., 13 Mar 2026). This suggests NanoVDR is best viewed as a deployment-oriented retriever that converts a strong but impractical VLM into an operational VDR system with minimal loss in quality.

In summary, NanoVDR is a text-side distillation framework for visual document retrieval that exploits query–document asymmetry. Its core technical claim is that direct cosine alignment to teacher query embeddings is superior to ranking-based distillation in this regime, and its main practical result is that a 69M text encoder can replace a 2B multimodal query pathway while retaining 95.1% of teacher quality and reducing CPU query latency by 50× (Liu et al., 13 Mar 2026).