Tevatron 2.0: Unified Document Retrieval Toolkit across Scale, Language, and Modality (2505.02466v1)

Abstract: Recent advancements in LLMs have driven interest in billion-scale retrieval models with strong generalization across retrieval tasks and languages. Additionally, progress in large vision-LLMs has created new opportunities for multimodal retrieval. In response, we have updated the Tevatron toolkit, introducing a unified pipeline that enables researchers to explore retriever models at different scales, across multiple languages, and with various modalities. This demo paper highlights the toolkit's key features, bridging academia and industry by supporting efficient training, inference, and evaluation of neural retrievers. We showcase a unified dense retriever achieving strong multilingual and multimodal effectiveness, and conduct a cross-modality zero-shot study to demonstrate its research potential. Alongside, we release OmniEmbed, to the best of our knowledge, the first embedding model that unifies text, image document, video, and audio retrieval, serving as a baseline for future research.

A Comprehensive Analysis of Tevatron 2.0: A Unified Document Retrieval Toolkit

The paper "Tevatron 2.0: Unified Document Retrieval Toolkit across Scale, Language, and Modality" presents an updated version of the Tevatron toolkit, aimed at facilitating large-scale, multilingual, and multimodal document retrieval. The authors introduce a unified pipeline for exploring retriever models spanning different scales, languages, and modalities. This research is of particular interest in the context of advancing retrieval technology, leveraging recent advancements in LLMs and large vision-LLMs.

Key Contributions

The Tevatron 2.0 represents significant improvements over its predecessor by integrating several advanced methodologies:

1. Unified Data Management: The toolkit provides a revised data management framework that accommodates diverse data modalities—text, image, video, and audio—without introducing complexity. This is achieved through a new data format that separates training queries from the corpus, storing only document IDs, which allows dynamic loading of document content.
2. GPU Memory Efficiency: The integration of LoRA (Low-Rank Adaptation), DeepSpeed ZeRO optimization, and FlashAttention considerably reduces the GPU memory requirements for training billion-scale retrieval models. These practices allow for efficient training even on limited computational resources, therefore democratizing research opportunities in the field of information retrieval.
3. Inference Efficiency: The incorporation of vLLM optimizes encoding speed and simplifies deployment of models, facilitating retrieval collaboration in frameworks like retrieval augmented generation. Moreover, the toolkit supports nested scalable representations à la Matryoshka Representation Learning (MRL) to dynamically adjust text embedding dimensionality.
4. Multimodal Retrieval Capability: The release of OmniEmbed—a pioneering embedding model—provides a robust baseline for text, image, video, and audio retrieval. The toolkit demonstrates effective multilingual and multimodal retrieval through a unified dense retriever, showcasing strong generalization potential across tasks.

Empirical Evaluation

The empirical analysis underscores the effectiveness of the Tevatron toolkit:

• Multilingual and Multimodal Effectiveness: The unified dense retriever yielded competitive results across multiple evaluation benchmarks, including BEIR, ViDoRe, and MIRACL. Such benchmarks cover a wide array of retrieval tasks across English, multiple languages, and various modalities.
• Zero-Shot Generalization: Notably, the Tevatron-BGE model variant trained on text-only retrieval data showed promising cross-modal zero-shot effectiveness, surpassing models optimized for specific modalities, and demonstrating the modality-agnostic prowess of the vision-LLM backbone when trained with diverse text retrieval tasks.
• Video and Audio Retrieval: The Tevatron-Omni model effectively engaged in video and audio retrieval, indicating potential for broader applications beyond conventional text and image retrieval scenarios.

Implications and Future Directions

The implications of Tevatron 2.0 are profound, presenting a versatile toolkit for both academia and industry applications. It enables researchers to prototype rapidly, explore novel combinations of retrieval tasks, languages, and modalities, and to push the boundaries of unified retrieval systems.

Looking forward, the research and development in scalable, diverse, and efficient retrieval systems is likely to benefit from the open-source nature of Tevatron 2.0. Potential future directions include further optimization of training and inference methodologies, greater exploration of cross-modal retrieval capabilities, and expanded support for dynamic and heterogeneous datasets.

In conclusion, Tevatron 2.0 exemplifies a crucial advancement in document retrieval technology, aligning with current trends towards multimodal integration and providing substantial infrastructure for future innovations in the retrieval field. The toolkit stands poised as a critical resource for advancing retrieval model capabilities across scale, language, and modality.