Papers
Topics
Authors
Recent
Search
2000 character limit reached

The Commercial Tax: Rent-vs-Own Blind Spots in Multi-Hop Retrieval Benchmarks

Published 17 Aug 2026 in cs.IR and cs.CL | (2608.16096v1)

Abstract: Enterprises connect LLMs to their own data through retrieval. The benchmarks that rank multi-hop retrieval systems leave out two facts a buyer needs before a published number can be used: whether the retrieval backbone may be deployed commercially, and what it costs to build. On licensing: the field's dense-retrieval anchor, NV-Embed-v2, is licensed cc-by-nc-4.0. Of the four leading MuSiQue systems we audit (HippoRAG-2, PropRAG, SAG, KET-RAG), three depend on it for their best numbers and none says so. On performance: we measure thirteen embedders from eight makers on one identical MuSiQue harness with bootstrap confidence intervals throughout. Until mid-2026 there was a real commercial tax: the best commercially-licensed embedder trailed the anchor by 2.31 Recall@5 points (95% CI [0.91, 3.71], p=0.001). NVIDIA's Nemotron-3-Embed-8B, released 2026-07-16, has closed it: +0.24 at Recall@5 (95% CI [-0.94, +1.43], p=0.69), -0.58 at Recall@10 (p=0.28). It matches the anchor, does not beat it, and is the only entrant that is commercially licensed, free to self-host, and indistinguishable from the anchor; every other entrant meeting the first two conditions sits 5.2 to 14.6 points below. The durable finding is the paid-versus-free divide: API embedders charge per token on every re-index, self-hosted ones charge nothing. On cost: three of five audited systems (adding Microsoft's GraphRAG) do not disclose indexing cost, and the only published GraphRAG dollar figures span 11x inside one third-party paper (USD 2.30 vs USD 24.94 to index a 5.64 MB corpus once); extrapolated to 1 TB that undisclosed choice separates roughly USD 428K from $4.6M. Our cost model keeps one-time embedding apart from recurring answering: at 1 TB, embedding sits 7.5x-900x below graph construction, and a year of answering at 10,000 queries/day sits 350x or more below it.

Authors (2)

Summary

  • The paper audits multi-hop RAG benchmarks and finds that NV-Embed-v2, the common retrieval anchor, is non-commercially licensed while three dependent systems omit this limitation in their disclosures.
  • The paper measures 13 embedders on the same 1,000-question MuSiQue harness and finds a 2.31-point commercial Recall@5 gap in June 2026 that narrowed to a statistically indistinguishable difference after Nemotron-3-Embed-8B’s release.
  • The paper shows that indexing costs are often undisclosed and configuration-sensitive, with GraphRAG estimates ranging from $2.30 to $24.94 on the same corpus and projected 1TB costs spanning roughly $428,000 to $4.6 million.

The problem the paper identifies

This paper argues that the benchmarks used to rank multi-hop retrieval-augmented generation (RAG) systems report headline numbers stripped of the two variables that determine whether a corporate buyer can actually use them: whether the retrieval backbone may be legally deployed commercially, and what it costs to build. The evaluation target is the MuSiQue closed-corpus protocol standardized by HippoRAG-2, under which a fixed pool of 11,656 Wikipedia passages and a 1,000-question sample define the comparison, with Recall@kk as the metric. Under this protocol, the dense-retrieval floor is NV-Embed-v2, and every leading system — HippoRAG-2 (74.7% Recall@5), PropRAG (78.3%), and SAG (80.0%, or 81.7% in its own ablation when swapped onto NV-Embed-v2) — reports numbers that inherit the properties of that single component.

The authors frame the problem by analogy to quantitative finance: a strategy backtest that omits transaction costs is not a wrong backtest, but it measures something the practitioner cannot trade. Similarly, a benchmark number measured under conditions the reader does not share — non-commercial licensing, undisclosed indexing cost — answers a question practitioners are not asking. The paper is explicit about its non-claims: it does not allege misconduct or invalid protocols, only that the published record omits conditions that determine deployability.

License verification and the anchor's lineage

NV-Embed-v2 is licensed cc-by-nc-4.0, and its model card states plainly that it "should not be used for any commercial purpose." The paper traces the restriction to a specific cause: the training mix includes MS MARCO, whose license is non-commercial. An NVIDIA representative confirmed on the model's HuggingFace discussion board that non-commercial training data is the reason the model could not be declared commercial. The paper notes that accessing the model through NVIDIA's paid NIM inference service changes who operates the hardware, not what the model is licensed for — a different access channel, not a separate license.

The disclosure audit then searches the full text and linked repositories of the three anchor-dependent systems (HippoRAG-2, PropRAG, SAG) for license terms. None discloses the restriction anywhere. The fourth audited system, KET-RAG, builds on OpenAI's text-embedding-3-small and is the counterexample showing a commercially usable embedder was a live option. The authors are careful to state that the license is not hidden — it is on the model card — so the finding is a failure to report a stated fact, not a concealed one. They also state plainly that the audited set is a relevance-seeded snowball sample, so all counts describe these papers and never a prevalence claim about the field.

The commercial-embedder retrieval floor

The paper measures thirteen embedders from eight makers on one identical harness: embed the 1,000 questions with no decomposition or reranking, embed the corpus once, retrieve by cosine similarity, score Recall@kk. All measurements, including NV-Embed-v2 itself, are run by the authors rather than borrowed from the literature — a choice justified by their finding that query-formatting choice alone swings one entrant (Qwen3-VL-Embedding-8B) by 11.6 Recall@5 points, making literature figures of unknown provenance unsafe to compare.

The headline result: NVIDIA's Nemotron-3-Embed-8B, released 2026-07-16, is statistically indistinguishable from the non-commercial anchor. The paired difference is +0.24+0.24 Recall@5 (95% CI [−0.94,+1.43][-0.94, +1.43], p=0.69p=0.69) and −0.58-0.58 at Recall@10 (p=0.28p=0.28). Before that release, the best commercially licensed embedder in the panel — Google Gemini embedding-001 at 67.24 — trailed the anchor by 2.31 Recall@5 points, a significant gap (95% CI [0.91, 3.71], p=0.001p=0.001). The "commercial tax" was therefore real and measurable in June 2026 and closed within a single release cycle in July 2026.

The paper is deliberate about the verb and about what the design cannot support. At n=1,000n=1{,}000, the smallest detectable difference at 80% power is roughly 1.7 points; a two-one-sided-test equivalence claim at a ±0.5\pm0.5-point margin would require roughly 33,700 questions given the observed point estimate. The supported claim is that any residual tax is small relative to the 2.31 points measured before, not that it is provably zero.

Inference is handled with unusual care. Because per-question scores across entrants are strongly correlated (mean kk0), overlap of marginal confidence intervals is formally invalid as a pairwise test. The uniqueness claim — that NV-Embed-v2 is the only entrant indistinguishable from the top entrant — is reported under Holm–Bonferroni correction across twelve comparisons and confirmed with Dunnett-type simultaneous intervals (max-kk1 critical value 2.78, independently re-implemented to within 0.4%). Under a global Holm family of all 47 paired tests in the paper, every claimed rejection stands.

The durable finding is the paid-versus-free divide rather than any single ranking. The four API-only providers charge per token on every re-index; self-hosted open-weight models charge nothing per token. Nemotron-3-Embed-8B is the only entrant simultaneously commercially licensed, free to self-host, and statistically indistinguishable from the anchor. Every other entrant meeting the first two conditions sits 5.2 to 14.6 points below it. Two secondary results reinforce the point that embedder-level measurement matters: Recall@10 lifts every embedder by a tightly clustered kk2 to kk3 points (all intervals exclude zero), and BGE-M3 — among the most widely deployed open-weight embedders at 35.6M monthly HuggingFace downloads — lands second-to-last in dense mode, evidence that general-purpose leaderboard reputation does not transfer to a stringently filtered hard benchmark.

Cost disclosure and standardized costing

The cost audit finds that three of five systems disclose no usable indexing cost: SAG and Microsoft's GraphRAG publish no dollar figure at all (SAG despite claiming production deployment at hundreds of millions of items), and HippoRAG-2 publishes token counts but no dollars. Only PropRAG (~$k$41.89) price the shared 5.64MB corpus.</p> <p>The sharpest finding concerns GraphRAG. KET-RAG&#39;s own tables price GraphRAG at $k$524.94 (high-performance) on the identical corpus — an 11x spread driven entirely by a chunk-size choice that Microsoft&#39;s paper does not name. An independent token-based reconstruction ($k$638.99) brackets the high-performance figure, so the two estimates agree once configuration is matched; what a reader cannot recover is which configuration &quot;the cost of GraphRAG&quot; means. Extrapolated linearly to 1TB, the undisclosed choice separates roughly $k$74.6M for a single index build. The authors caveat that linear extrapolation is almost certainly optimistic for graph construction, since entity extraction plausibly scales super-linearly, making the 1TB figures more likely underestimates.</p> <p>The cost model deliberately separates one-time embedding cost from recurring answering cost. At matched 1TB scale, embedding sits 7.5x to 900x below graph construction, and a full year of answering at 10,000 queries/day sits 350x or more below it. Answering cost was measured from real API usage with <a href="https://www.emergentmind.com/topics/gpt-4o-mini-8a7e420b-c7de-47e7-818a-da41dc130fa6" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">gpt-4o-mini</a> over actual retrieved passages: $k$80.060 per 1,000 queries at batch rates, with the rate card predicting the meter to within 4%, and the k=5→k=10 cost ratio tightly clustered at 1.86–1.93x across the panel. A practical crossover follows: at 10,000 queries/day, recurring answering cost overtakes one-time API re-embedding within the first year for corpora between roughly 10GB and 180GB. The paper's summary is pointed: embedding choice is a licensing and sovereignty question, answering cost is a volume question, and indexing architecture is where the money actually is — and disclosure runs inversely to size, since the largest cost line is the least reported.

Limitations and open questions

The paper concedes its limits with unusual specificity, several of which bear directly on the headline. The panel is not vendor-neutral: five of thirteen entrants are NVIDIA models, and the winning model shares a maker with the anchor it matches. The authors disclose this and point to the eight non-NVIDIA entrants as competitive context, but readers should weigh the headline accordingly. Uneven query-instruction sweeps flatter the anchor itself, which was swept over four instructions while Nemotron-3-Embed-8B was measured in one configuration — a bias that runs against the paper's own thesis and makes the $k$9 conservative. The pre-specification of the primary Gemini-vs-anchor comparison is asserted from version history, not external registration, and the authors say so plainly. A pilot on 2WikiMultihopQA replicates the match between the top two embedders but shows no measurable commercial gap there ($+0.24$0 points, $+0.24$1), weakening the magnitude claim: the tax's size is a property of the corpus, not a constant of the field. All Recall@$+0.24$2 figures are index-free ceilings — exhaustive cosine search, no ANN — so none is a production recall figure. Hosted endpoints are exposed to silent server-side drift (text-embedding-3-large moved $+0.24$3 points between rounds), making hosted figures point-in-time. The cost model prices machine time only, not human setup cost, and does not measure the query-time retrieval-LLM costs of the audited systems, a gap none of them disclose either.

Conclusion

This paper measures two omissions in the multi-hop retrieval literature and finds both consequential. On licensing, the protocol's anchor is non-commercially licensed for a traceable reason, and none of the three anchor-dependent systems discloses it. On cost, the literature's least-disclosed number is its largest: graph construction, which reaches six and seven figures at institutional scale with an 11x configuration-dependent uncertainty for the field's most-cited baseline. Measured on one harness, the quality penalty for requiring a commercial license was a significant 2.31 Recall@5 points in June 2026 and is no longer measurable as of July 2026 — with the explicit caveat that "matches" is the strongest verb the data support, and that the finding holds on one corpus. The durable contribution is methodological as much as empirical: the released per-question vectors, harness, and cost model make both the ranking and the disclosure audit recomputable by any reader, which is the standard the paper argues the field should adopt.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Tweets

Sign up for free to view the 2 tweets with 6 likes about this paper.