---
title: 'Clibib: Deterministic BibTeX Retrieval Tool'
url: https://www.emergentmind.com/topics/clibib
type: topic
---

# Clibib: Deterministic BibTeX Retrieval Tool

Searching arXiv for papers on clibib and its evaluation.
clibib is an open-source Python tool for deterministic BibTeX retrieval designed to mitigate citation hallucinations in LLM-based scientific writing workflows. It is introduced as a retrieval layer for scientific publishing agents that resolves persistent identifiers such as DOI, arXiv ID, ISBN, and PMID through the Zotero Translation Server, with CrossRef fallback when needed, and returns BibTeX exported from upstream registries rather than reconstructed by a model [2604.03159]. In the evaluation accompanying its release, clibib is situated within a broader study of BibTeX generation by search-enabled frontier models, where field-level accuracy reached 83.6% but only 50.9% of entries were fully correct, motivating deterministic metadata retrieval as a distinct systems intervention rather than a prompting refinement [2604.03159].

## 1. Definition and scope

clibib is presented as an open-source Python tool, approximately 230 lines, MIT-licensed, released on PyPI and as an agentskills.io-compatible skill, whose purpose is deterministic BibTeX retrieval [2604.03159]. Its central design premise is that bibliographic metadata should not be generated when authoritative values already exist in publisher and registry databases. In this setting, “deterministic retrieval” means that, given a persistent identifier, the tool resolves to exactly one authoritative record without ranking, heuristic matching, or any LLM involvement [2604.03159].

The paper distinguishes sharply between identifier-based resolution and ambiguous textual lookup. When the query is a DOI, arXiv ID, ISBN, or PMID, clibib uses the Zotero Translation Server’s identifier resolvers so that a single metadata record is returned, and then converts that record to BibTeX through Zotero’s export endpoint [2604.03159]. By contrast, when the input is free text or another ambiguous query such as a plain title, determinism cannot be guaranteed; candidate ranking is then required [2604.03159]. This suggests that clibib is best understood not as a general bibliographic search engine but as a metadata finalization layer whose strongest guarantees depend on upstream identifier discovery.

The tool is evaluated in the context of LLM-based scientific publishing agents, not as a standalone bibliographic manager. Its role is to replace the “generative last-mile” for structured citation fields, where models otherwise invent, truncate, transpose, or substitute metadata [2604.03159].

## 2. Retrieval model and system design

clibib’s design goals are stated as determinism when possible, breadth via aggregation, and practical robustness [2604.03159]. Breadth is achieved by leveraging the Zotero Translation Server, which aggregates CrossRef, DBLP, PubMed, and other sources, and by falling back to the CrossRef API when Zotero returns no candidates [2604.03159]. Practical robustness includes URL normalization, preference for identifier resolution over page scraping, and rate limiting to remain service-friendly [2604.03159].

The resolution logic is organized around input type. Persistent identifiers are routed to Zotero’s `/search` endpoint for deterministic resolution; DOI-style URLs are parsed and likewise routed to `/search` rather than scraped; other URLs are normalized and resolved via Zotero’s `/web`; titles and other text queries invoke ranked candidate selection [2604.03159]. When Zotero `/search` returns an empty response, including the cases described as HTTP 300 multiple-candidate ambiguity or no candidates, clibib queries `api.crossref.org/works` with the original text and ranks up to 10 candidates by token Jaccard similarity with a substring tiebreaker [2604.03159].

The paper emphasizes that clibib does not synthesize fields. It “faithfully exports what Zotero’s resolvers return,” and the presence or absence of particular BibTeX fields depends on publisher registry coverage for the resolved record [2604.03159]. Consequently, clibib’s guarantees are about provenance and retrieval path, not schema completion. A plausible implication is that clibib improves reliability primarily for fields that are strongly standardized in upstream databases, especially identifiers and numeric publication metadata.

## 3. Inputs, normalization, and version handling

clibib supports four major input categories: persistent identifiers, URLs, titles, and other text queries [2604.03159]. DOI, arXiv ID, ISBN, and PMID are the preferred path because they enable deterministic resolution. DOI-style URLs are parsed into identifiers. Other URLs are sent through Zotero’s web translation path, which is scraping-based rather than identifier-based. Titles and free-text queries rely on ranked retrieval and therefore lose determinism [2604.03159].

A substantial part of the surrounding benchmark concerns version multiplicity. The evaluation constructs version-aware ground truth across arXiv preprints, proceedings, and journal versions, discovered via OpenAlex locations and URL or source-type heuristics [2604.03159]. Across 997 initially sampled papers, 1,131 fetchable versions were found, and 13.4% of papers had multiple citable versions, rising to 44.8% in Quantum Computing [2604.03159]. clibib itself does not impose a version-preference policy; it returns the record corresponding to the identifier or page it is given. If a workflow must prefer a particular version, such as journal DOI over preprint, that policy must be implemented upstream by choosing the identifier passed to clibib [2604.03159].

For evaluation, outputs were normalized along several field-specific rules: authors by first-author last name in Stage 1 with stricter all-author analyses also examined; titles lowercased with LaTeX commands and braces removed; venues normalized through a 141-variant synonym table across 40 target venues; DOIs lowercased with URL prefixes stripped; pages normalized to start–end; and years matched exactly as four digits [2604.03159]. These normalization rules belong to the benchmark rather than the retrieval mechanism itself, but they are essential to interpreting reported gains.

## 4. Integration architectures in publishing agents

The paper evaluates two integration patterns: a single-stage tool loop and a two-stage “search then revise” pipeline [2604.03159]. The contrast between them is one of the central contributions of the work.

In the single-stage integration, the LLM has access both to a web-search tool and to `clibib_lookup`. The model searches the web, identifies an identifier or landing page, invokes clibib, and emits the final BibTeX entry [2604.03159]. This arrangement improves accuracy but still allows the model to interleave search, tool use, and metadata revision in ways that produce regressions.

In the two-stage integration, Stage 1 has the model produce a baseline entry using native web search without clibib. Stage 2 is controlled by the client rather than the model: clibib is called using a strict priority order—URL if present, else DOI, else title—and the returned record is accepted only if a title-match gate passes, where lowercased, tokenized titles with stopword filtering must have Jaccard similarity at least 0.3 [2604.03159]. If the title match passes, the model receives both entries in a targeted revision prompt and replaces authors, title, year, venue, volume, number, pages, and DOI with database values, keeping baseline values only for fields the database lacks; if the titles do not match, the baseline is preserved unchanged [2604.03159].

The study argues that separating search from revision reduces instruction overload and lowers regressions independently of model capability [2604.03159]. This suggests that clibib’s effectiveness is partly architectural: the same retrieval primitive can perform differently depending on whether it is embedded inside a tool-using generation loop or used as a post hoc correction layer.

## 5. Benchmark, metrics, and empirical performance

clibib is evaluated on a benchmark of 931 papers across Artificial Intelligence, Medicine, Materials Science, and Quantum Computing, stratified into popular, low-citation, and recent tiers [2604.03159]. Three search-enabled models—GPT-5, Claude Sonnet-4.6, and Gemini-3 Flash—generate one BibTeX entry per paper with web search enabled [2604.03159]. The evaluation scores nine fields per entry: `entry_type`, `author`, `title`, `year`, `venue`, `volume`, `number`, `pages`, and `doi`, while `entry_key` is always marked not applicable because citation keys are arbitrary [2604.03159].

Baseline performance establishes the problem setting. Overall field-level accuracy is 83.6%, but only 50.9% of entries are fully correct [2604.03159]. Accuracy drops by 27.7 percentage points from popular papers at 92.7% to recent papers at 65.0%, which the paper interprets as evidence of heavy reliance on parametric memory even when search is available [2604.03159].

The principal results for clibib are as follows:

| Setting | Field-level accuracy | Fully correct entries | Regression rate |
|---|---:|---:|---:|
| Baseline search-enabled LLMs | 83.6% | 50.9% | — |
| Single-stage clibib | 85.3% | GPT-5: 69.2%; Claude: 70.4% | 4.8% |
| Two-stage clibib | 91.5% | 78.3% | 0.8% |

In the single-stage setting, pooled accuracy for GPT-5 and Claude rises from 80.9% to 85.3%, and the fraction of fully correct entries rises from 46.1% to 69.2% for GPT-5 and from 53.0% to 70.4% for Claude [2604.03159]. The largest field gains are `number` at +19.1 percentage points, `volume` at +9.0, `pages` at +7.9, and `doi` at +7.2 [2604.03159].

In the two-stage setting, pooled field-level accuracy rises from 83.6% to 91.5%, fully correct entries rise from 50.9% to 78.3%, and regression is held to 0.8% [2604.03159]. Per-model improvements are GPT-5 from 81.4% to 92.5%, Claude from 80.4% to 87.2%, and Gemini from 88.9% to 94.9% [2604.03159]. The largest pooled field gains are `number` at +22.1 percentage points, `doi` at +14.9, and `pages` at +13.0 [2604.03159]. These improvements are concentrated in structured numeric and identifier fields, a pattern consistent with the tool’s reliance on registry data rather than generative reconstruction.

The paper formalizes the principal metrics as
$$
A = \frac{\sum_{i=1}^{N} \sum_{f=1}^{F} \mathbf{1}[\text{field}_{i,f}\ \text{is correct}]}{N \times F},
$$
for overall field-level accuracy,
$$
FCR = \frac{\left|\left\{ i : \sum_{f=1}^{F} \mathbf{1}[\text{field}_{i,f}\ \text{is correct}] = F \right\}\right|}{N},
$$
for fully correct entry rate, and
$$
\rho = \frac{R}{N_{\text{prev-correct}}}\times 100\%,
$$
for regression rate [2604.03159]. The emphasis on regression is important: the paper treats a mitigation layer as successful not merely when it fixes errors, but when it rarely corrupts already correct fields.

## 6. Error structure, limitations, and operational guidance

The study defines a six-way field-level error taxonomy: Correct, Missing, Fabricated, Partially correct, Substituted, and Not applicable [2604.03159]. It also introduces an error co-occurrence matrix,
$$
C_{ab} = P(\text{field } b \text{ wrong} \mid \text{field } a \text{ wrong}) = \frac{|E_a \cap E_b|}{|E_a|},
$$
to characterize failure structure [2604.03159]. Two major error modes emerge. The first is wholesale entry substitution, in which identity fields fail together; for example, \(P(\text{title wrong} \mid \text{author wrong}) = 0.71\), \(P(\text{year wrong} \mid \text{author wrong}) = 0.74\), \(P(\text{venue wrong} \mid \text{author wrong}) = 0.77\), \(P(\text{doi wrong} \mid \text{year wrong}) = 0.91\), and \(P(\text{doi wrong} \mid \text{venue wrong}) = 0.98\) [2604.03159]. The second is isolated field error, especially in formatting and numeric metadata, where co-error rates are lower [2604.03159].

Among 1,372 entries with at least one error, 65.2% are isolated, 33.7% are mixed, and only 1.0% are wholesale substitutions with at least three substituted fields [2604.03159]. Two-stage clibib fixes 57.9% of isolated-error entries to fully correct but only 14.3% of wholesale substitutions [2604.03159]. This is an important limitation: once the wrong paper has effectively been selected, deterministic revision cannot reconstruct the intended citation if the database path also reflects that wrong target.

Other limitations are coverage and ambiguity. Zotero and CrossRef indexation lags or paywalls can yield `not_found`, described as approximately 11% of lookups during ground-truth construction, and single-stage success rates vary by domain, from 97.2% in Quantum Computing to 81.8% in AI [2604.03159]. Medicine is specifically noted as being affected by paywalls [2604.03159]. Title-based ambiguity remains a risk when identifiers are absent, even though clibib mitigates it by Jaccard ranking and the two-stage pipeline further gates revision on title similarity [2604.03159]. The paper also notes that normalization rules and synonym tables target English and Latin scripts, and do not extend evaluation to non-English venues [2604.03159].

Operationally, the recommended pattern is retrieval-centric “search then revise” integration [2604.03159]. The preferred query order is to use stable identifiers surfaced by search telemetry—DOI if available, otherwise publisher or arXiv URL, otherwise title [2604.03159]. The system should route identifiers to Zotero `/search`, DOI-style URLs to parsed `/search`, other URLs to Zotero `/web`, and only then to CrossRef fallback if `/search` returns no candidates [2604.03159]. Before revision, a title Jaccard threshold should be applied; the evaluated two-stage system used a threshold of at least 0.3 [2604.03159]. The paper further notes that clibib enforces polite behavior with 2 requests per second and normalizes known URL patterns such as arXiv PDF or HTML to the abstract page, and alphaxiv or certain HuggingFace paper links to arXiv [2604.03159].

In this form, clibib functions as a narrow but effective mitigation layer for structured citation metadata. Its contribution lies less in bibliographic discovery than in enforcing authoritative resolution and in demonstrating that integration architecture—particularly the separation of search from revision—materially affects the reliability of LLM-based scientific writing systems [2604.03159].

Source: https://www.emergentmind.com/topics/clibib