---
title: 'OneDB: Distributed Multi-Metric Retrieval'
url: https://www.emergentmind.com/topics/onedb
type: topic
---

# OneDB: Distributed Multi-Metric Retrieval

Searching arXiv for the specified OneDB-related papers to ground the article.
arxiv_search.query{"search_query":"id:2507.04256 OR ti:\"OneDB: A Distributed Multi-Metric Data Similarity Search System\" OR ti:Poseidon: A OneGraph Engine OR ti:\"Machine Learning with DBOS\"","max_results":10}
arxiv_search.query{"search_query":"all:\"OneDB: A Distributed Multi-Metric Data Similarity Search System\"","max_results":5}
arxiv_search.query 
Searching for the main OneDB paper and closely related systems papers on arXiv.
OneDB is a distributed multi-metric data similarity retrieval system in which each modality is modeled as its own metric space and multi-modal retrieval is performed by combining per-modality distances through a weighted multi-metric distance. Implemented on Spark, it supports exact multi-metric range queries and exact multi-metric \(k\)NN queries over heterogeneous data that may include low-dimensional vectors, high-dimensional vectors or embeddings, strings or text, geospatial coordinates, and numeric attributes [2507.04256]. In adjacent systems literature, the same term also appears in a broader architectural sense, referring to a database-centric unification direction rather than this specific similarity-search engine [2510.11166][2208.05101].

## 1. Scope, motivation, and conceptual position

OneDB is designed for retrieving similar objects when each object has multiple heterogeneous attributes and similarity depends on several modalities jointly. The motivating settings include health care, e-commerce, and rentals or apartments, where image similarity, text similarity, numeric similarity, and spatial proximity carry complementary information. The system is positioned against two insufficient alternatives: multi-vector systems that use vector fusion or isolated search, and prior metric-space systems that usually support only a single metric space or remain standalone rather than distributed [2507.04256].

A central distinction is that OneDB is metric-centric rather than vector-centric. Vector or embedding representation is only one possible representation for some modalities. Metric representation is broader: any modality that has a valid distance metric can participate. The paper explicitly contrasts this with vector databases and argues that OneDB is not restricted to vectors. This directly addresses a common misconception that OneDB is merely a multi-vector retrieval layer. Its abstraction is broader, and its query semantics are exact rather than approximate.

Another important misconception is that OneDB is a general-purpose unified DBMS in the same sense as a relational engine. The named 2025 system is instead a distributed exact similarity-search system for multi-metric spaces. A plausible implication is that the “One” in the system name refers to unifying heterogeneous similarity spaces under one retrieval model, not to replacing every database subsystem with a single physical engine.

## 2. Formal model and retrieval semantics

The formal basis of OneDB is a multi-metric space. A metric space is defined as a pair \((M, \delta)\), where \(M\) is a domain of objects and \(\delta\) satisfies symmetry, non-negativity, identity, and triangle inequality. A multi-metric space is then defined as \((\mathcal{M}, \Delta)\), where \(\mathcal{M}\) is a collection of domains \(M_i\) and \(\Delta\) is the corresponding set of distance metrics \(\delta_i\). An object is represented as
$$
o^M = \{o^i \mid 1 \leq i \leq m\},
$$
where \(o^i \in M_i\) is the component in the \(i\)-th modality [2507.04256].

Each modality uses its own distance function. The paper gives \(L_1\)-norm for some numeric or vector modalities, \(L_2\)-norm for spatial coordinates, and edit distance for strings or text. Distances are normalized by dividing them by twice the median of all observed distances in the corresponding metric space. The weighted multi-metric distance is defined as
$$
\delta_W(\mathbf{q}^M, \mathbf{o}^M) = \sum_{\delta_i \in \Delta} \omega_i \cdot \delta_i(q^i, o^i),
$$
where \(W = (\omega_1, \ldots, \omega_M)\), \(\omega_i \in [0,1]\), and larger \(\omega_i\) means modality \(i\) contributes more [2507.04256].

The system defines two exact retrieval operators. The multi-metric range query is
$$
\text{MMRQ}(q^M, W, r) = \{o^M \mid o^M \in S \text{ and } \delta^W(q^M, o^M) \leq r\},
$$
and the multi-metric \(k\)NN query is
$$
\text{MMkNNQ}(q^M, W, k) = \{S' \mid S' \subseteq S, \ |S'| = k, \ \text{and } \forall s^M \in S'( \ \forall o^M \in (S - S')( \ \delta^W(q^M, s^M) \leq \delta^W(q^M, o^M)))\}.
$$
The paper explicitly states \(\omega_i \in [0,1]\) and does not explicitly impose \(\sum_i \omega_i = 1\). This matters because the implementation permits arbitrary nonnegative weights in that interval rather than requiring simplex normalization.

## 3. Query interface and modality-weight learning

OneDB extends Spark SQL with exact similarity-search operators. The two query forms are:

```sql
SELECT * FROM T WHERE T.col IN ODBRANGE(q^M, W, r)
```

and

```sql
SELECT * FROM T WHERE T.col IN ODBKNN(q^M, W, k)
```

Because these operators are embedded in Spark SQL, they can be combined with normal SQL predicates and relational operations, and they inherit full structured query support from Spark SQL [2507.04256].

The modality weight vector \(W\) can be supplied manually or learned. If users do not know appropriate weights, OneDB uses a lightweight multi-metric weight learning model based on contrastive learning. Users provide a set of \(N\) query points and their corresponding true \(k\)-nearest neighbors. The system then iteratively generates training samples by running \(\text{MMkNNQ}(q^M, \bar{W}, k)\) under the previous iteration’s weight vector \(\bar{W}\). The intended effect is to reduce \(\delta_W(q^M,o^{M+})\) for positives and increase \(\delta_W(q^M,o^{M-})\) for negatives, with optimization by gradient descent [2507.04256].

The learning module is deliberately lightweight rather than a heavy representation-learning subsystem. It learns relative weights across modalities instead of learning embeddings from scratch. The reported efficiency is that only 30 historical query points are used, and training reaches 90% recall under 100 seconds. The hard-negative strategy outperforms random negative sampling in convergence and recall.

The paper also contains explicit inconsistencies in this part of the formalization. The prose says positive samples are the intersection of true \(k\)NNs and retrieved \(k\)NNs, while the displayed formula shows a union. The printed loss equation is also malformed and semantically questionable as printed. These issues do not negate the high-level mechanism, but they do leave some implementation details underspecified.

## 4. Dual-layer indexing and exact search algorithms

The system’s central systems contribution is a dual-layer indexing design. On the master node, OneDB builds a global Reference-\(R^*\)-tree (\(RR^*\)-tree). For each metric space \(M_i\), one pivot \(p_i\) is selected using FFT (farthest-first traversal). Each object \(o^M\) is mapped to an \(m\)-dimensional vector
$$
[\delta_1(p_1^1,o^1),\delta_2(p_2^2,o^2),\ldots,\delta_m(p_m^m,o^m)],
$$
and an \(R^*\)-tree is built over these mapped vectors. The paper explicitly states that only one pivot per modality is used to avoid high global dimensionality [2507.04256].

At the worker side, each partition is decomposed by modality and indexed separately. The supported local indexes are R-tree for low-dimensional vector data, MVP-tree for high-dimensional vector data, and inverted index for text. Automatic local index choice uses the estimated hidden dimension from mean pairwise distance \(\mu\) and distance variance \(\sigma^2\); if the modality is text, the system uses an inverted index, else if the hidden dimension is greater than \(5\), it uses MVP-tree, and otherwise it uses R-tree.

Search uses pruning at both layers. For a range query, the master maps the query to a pivot-embedded bounding region
$$
R_q = [\delta_i(q^i, p_i^i)-r, \delta_i(q^i, p_i^i)+r \mid 1 \le i \le m].
$$
The paper’s global pruning lemma states that if, for some relevant modality, \(R_q[i]\) does not intersect with a non-leaf entry’s \(e.MBR[i]\), the entry can be pruned. At the worker, the local pruning lemma states that an object can be included in the result set of \(\text{MMRQ}(q^M, W, r)\) if there exists at least one metric with \(\omega_i > 0\) such that
$$
\delta_i(q^M, o^M) \leq \frac{r}{\sum_{\delta_i \in \Delta} \omega_i}.
$$
The paper says this follows from the pigeonhole principle.

For exact multi-metric \(k\)NN, OneDB does not merge top-\(k\) answers from each modality. Instead, it uses a two-phase strategy. It first estimates an upper bound \(\bar{dis_k}\) for the true \(k\)-th nearest-neighbor distance from a sampled region, then converts the \(k\)NN problem into an exact range query with radius \(\bar{dis_k}\). Because \(\bar{dis_k}\) is an upper bound on the true \(k\)-th neighbor distance, the exact top-\(k\) answers must be contained in the resulting candidate set. Final verification and ranking are then exact.

## 5. Evaluation, datasets, and quantitative results

The experimental evaluation uses three real datasets and two synthetic datasets.

| Dataset | Objects | \(m\) |
|---|---:|---:|
| Air | 1,150,000 | 13 |
| Food | 38,757 | 9 |
| Rental | 113,176 | 5 |
| Synthetic | 200,000 | 50 |
| Synthetic II | 10,000,000 | 96 |

The baselines are DESIRE-D, DIMS-M, and Milvus, with additional OneDB-R2M and OneDB-MVP2M ablations that replace local index choices [2507.04256].

The paper reports several headline results. Relative to Milvus, OneDB improves retrieval accuracy by 12.63\%–30.75\% at comparable efficiency. Milvus may run slightly faster at low query ratios, but only at about 70% recall; when pushed toward 85% recall, Milvus becomes about 10% slower than OneDB. Relative to state-of-the-art single- or multi-metric solutions, OneDB achieves 2.5x–5.75x speedup. As worker count increases, query time decreases for all methods, and OneDB remains best by 1.7x–4.8x. Workload standard deviation rises as the cluster grows, but OneDB remains more balanced than DESIRE-D and slightly better than DIMS-M.

Construction costs are also reported explicitly. For Air, OneDB build time is 743.84s versus 1209.36s for DIMS-M and 893.52s for DESIRE-D. For Synthetic II, OneDB is 2083s versus 2359s for DIMS-M and 2135s for DESIRE-D. Update costs for random updates of 0.1%–1.0% remain low, in milliseconds, and query latency increases only slightly. For example, at 0.1% update ratio, Food has 8.93 ms average update cost with +0.53 ms query time, and Air has 39.06 ms with +4.73 ms. At 1.0% update ratio, Food has 9.52 ms with +0.92 ms, and Air has 42.22 ms with +6.32 ms.

The autotuning subsystem uses DDPG and reports that after about 50 training steps, RL-based tuning yields 15%–17% performance improvement. The exponential reward variant gives the best final latency and throughput. The paper also reports an ablation in which replacing MVP-tree with M-tree incurs up to 25% higher query latency.

## 6. Broader architectural usage of the term

In the supplied literature, “OneDB” is not only the name of the distributed multi-metric retrieval system. It also denotes a broader architectural direction centered on collapsing multiple data and execution layers into one database-centric substrate. Poseidon, the engine behind Neptune Analytics, is explicitly described as a graph-specific prototype of a broader “OneDB” direction: one engine, one storage substrate, one optimizer or execution stack, and one durability story across multiple graph representations and workload types [2510.11166].

In that broader usage, the unification target is not multi-modal similarity search but system architecture. Poseidon unifies RDF and labeled property graph data through a 1G model over \(S, P, O, I\), uses one identifier regime below the logical layer, supports declarative graph querying and algorithm invocation in one plan space, and combines an in-memory partitioned storage engine, secondary indices, MVCC, logical logging, and a vectorized dataflow execution engine. The paper explicitly states that Poseidon is not a general multimodel OneDB yet; it is a graph-native OneGraph engine. This distinction is important because it prevents the broader architectural slogan from being conflated with the named similarity-search system.

DBOS is relevant in a related but separate way. It is described as a database-oriented operating system stack in which all system and application state are stored in database tables and operations on that state execute as transactions. The paper argues for collapsing OS state, app logic, workflow, metadata, provenance, and ML execution into a DBMS-centered stack, using VoltDB as the transactional DBMS and Vertica as an OLAP or provenance store [2208.05101]. This suggests that “OneDB” can also function as an architectural shorthand for a single transactional data and control plane, even when the actual implementation remains poly-engine.

A plausible implication is that the term has acquired two technically distinct meanings: a specific distributed exact multi-metric retrieval engine, and a broader unification thesis in systems design. The literature supplied here supports both usages, but they should not be treated as interchangeable.

## 7. Limitations, caveats, and open questions

The OneDB similarity-search paper states several limitations explicitly. Some formulas and definitions are inconsistent: positive-sample generation text says intersection while the displayed formula shows union, the loss function is malformed, and the “logarithmic” reward formula is not actually logarithmic. The learned weights depend on users providing a small set of representative query points and desired top-\(k\) results, so a single learned weight profile may be insufficient if user intent changes rapidly or is highly query-specific. The global \(RR^*\)-tree ignores user-specific weights by design, which preserves flexibility but may weaken global pruning relative to a weight-aware global index [2507.04256].

Current local index support is limited to R-tree, MVP-tree, and inverted index. Exact search may also be less attractive for workloads where approximate ANN is acceptable and latency is the only priority. Dynamic data is only lightly addressed: the evaluation reports low update cost, but the conclusion explicitly says future work will explore dynamic data scenarios more deeply. The MM\(k\)NN method also depends on estimating a good upper bound \(\bar{dis_k}\); if that bound is loose, the converted range query may admit many candidates and increase verification cost.

The broader “OneDB” architectural usage has its own caveats. Poseidon is explicitly not a general multimodel OneDB, and DBOS relies on external processes for GPU-backed ML while splitting transactional and analytical storage across VoltDB and Vertica [2510.11166][2208.05101]. This suggests that the unification ideal remains partially realized even in systems that strongly align with it. The practical lesson is that “OneDB” names both an implemented exact multi-metric retrieval system and an unfinished systems aspiration toward tighter integration of data, execution, provenance, and heterogeneous workloads.

Source: https://www.emergentmind.com/topics/onedb