Clibib: Deterministic BibTeX Retrieval Tool

• Clibib is an open-source Python tool that deterministically retrieves BibTeX by resolving persistent identifiers (DOI, arXiv ID, ISBN, PMID) without generative synthesis.
• It leverages the Zotero Translation Server and CrossRef fallback to aggregate authoritative metadata, achieving up to 91.5% field-level accuracy in two-stage integration.
• Evaluated within LLM-based scientific publishing workflows, clibib effectively minimizes citation hallucinations and provides robust, reliable citation metadata.

Searching arXiv for papers on clibib and its evaluation. clibib is an open-source Python tool for deterministic BibTeX retrieval designed to mitigate citation hallucinations in LLM-based scientific writing workflows. It is introduced as a retrieval layer for scientific publishing agents that resolves persistent identifiers such as DOI, arXiv ID, ISBN, and PMID through the Zotero Translation Server, with CrossRef fallback when needed, and returns BibTeX exported from upstream registries rather than reconstructed by a model (Rao et al., 3 Apr 2026). In the evaluation accompanying its release, clibib is situated within a broader study of BibTeX generation by search-enabled frontier models, where field-level accuracy reached 83.6% but only 50.9% of entries were fully correct, motivating deterministic metadata retrieval as a distinct systems intervention rather than a prompting refinement (Rao et al., 3 Apr 2026).

1. Definition and scope

clibib is presented as an open-source Python tool, approximately 230 lines, MIT-licensed, released on PyPI and as an agentskills.io-compatible skill, whose purpose is deterministic BibTeX retrieval (Rao et al., 3 Apr 2026). Its central design premise is that bibliographic metadata should not be generated when authoritative values already exist in publisher and registry databases. In this setting, “deterministic retrieval” means that, given a persistent identifier, the tool resolves to exactly one authoritative record without ranking, heuristic matching, or any LLM involvement (Rao et al., 3 Apr 2026).

The paper distinguishes sharply between identifier-based resolution and ambiguous textual lookup. When the query is a DOI, arXiv ID, ISBN, or PMID, clibib uses the Zotero Translation Server’s identifier resolvers so that a single metadata record is returned, and then converts that record to BibTeX through Zotero’s export endpoint (Rao et al., 3 Apr 2026). By contrast, when the input is free text or another ambiguous query such as a plain title, determinism cannot be guaranteed; candidate ranking is then required (Rao et al., 3 Apr 2026). This suggests that clibib is best understood not as a general bibliographic search engine but as a metadata finalization layer whose strongest guarantees depend on upstream identifier discovery.

The tool is evaluated in the context of LLM-based scientific publishing agents, not as a standalone bibliographic manager. Its role is to replace the “generative last-mile” for structured citation fields, where models otherwise invent, truncate, transpose, or substitute metadata (Rao et al., 3 Apr 2026).

2. Retrieval model and system design

clibib’s design goals are stated as determinism when possible, breadth via aggregation, and practical robustness (Rao et al., 3 Apr 2026). Breadth is achieved by leveraging the Zotero Translation Server, which aggregates CrossRef, DBLP, PubMed, and other sources, and by falling back to the CrossRef API when Zotero returns no candidates (Rao et al., 3 Apr 2026). Practical robustness includes URL normalization, preference for identifier resolution over page scraping, and rate limiting to remain service-friendly (Rao et al., 3 Apr 2026).

The resolution logic is organized around input type. Persistent identifiers are routed to Zotero’s /search endpoint for deterministic resolution; DOI-style URLs are parsed and likewise routed to /search rather than scraped; other URLs are normalized and resolved via Zotero’s /web; titles and other text queries invoke ranked candidate selection (Rao et al., 3 Apr 2026). When Zotero /search returns an empty response, including the cases described as HTTP 300 multiple-candidate ambiguity or no candidates, clibib queries api.crossref.org/works with the original text and ranks up to 10 candidates by token Jaccard similarity with a substring tiebreaker (Rao et al., 3 Apr 2026).

The paper emphasizes that clibib does not synthesize fields. It “faithfully exports what Zotero’s resolvers return,” and the presence or absence of particular BibTeX fields depends on publisher registry coverage for the resolved record (Rao et al., 3 Apr 2026). Consequently, clibib’s guarantees are about provenance and retrieval path, not schema completion. A plausible implication is that clibib improves reliability primarily for fields that are strongly standardized in upstream databases, especially identifiers and numeric publication metadata.

3. Inputs, normalization, and version handling

clibib supports four major input categories: persistent identifiers, URLs, titles, and other text queries (Rao et al., 3 Apr 2026). DOI, arXiv ID, ISBN, and PMID are the preferred path because they enable deterministic resolution. DOI-style URLs are parsed into identifiers. Other URLs are sent through Zotero’s web translation path, which is scraping-based rather than identifier-based. Titles and free-text queries rely on ranked retrieval and therefore lose determinism (Rao et al., 3 Apr 2026).

A substantial part of the surrounding benchmark concerns version multiplicity. The evaluation constructs version-aware ground truth across arXiv preprints, proceedings, and journal versions, discovered via OpenAlex locations and URL or source-type heuristics (Rao et al., 3 Apr 2026). Across 997 initially sampled papers, 1,131 fetchable versions were found, and 13.4% of papers had multiple citable versions, rising to 44.8% in Quantum Computing (Rao et al., 3 Apr 2026). clibib itself does not impose a version-preference policy; it returns the record corresponding to the identifier or page it is given. If a workflow must prefer a particular version, such as journal DOI over preprint, that policy must be implemented upstream by choosing the identifier passed to clibib (Rao et al., 3 Apr 2026).

For evaluation, outputs were normalized along several field-specific rules: authors by first-author last name in Stage 1 with stricter all-author analyses also examined; titles lowercased with LaTeX commands and braces removed; venues normalized through a 141-variant synonym table across 40 target venues; DOIs lowercased with URL prefixes stripped; pages normalized to start–end; and years matched exactly as four digits (Rao et al., 3 Apr 2026). These normalization rules belong to the benchmark rather than the retrieval mechanism itself, but they are essential to interpreting reported gains.

4. Integration architectures in publishing agents

The paper evaluates two integration patterns: a single-stage tool loop and a two-stage “search then revise” pipeline (Rao et al., 3 Apr 2026). The contrast between them is one of the central contributions of the work.

In the single-stage integration, the LLM has access both to a web-search tool and to clibib_lookup. The model searches the web, identifies an identifier or landing page, invokes clibib, and emits the final BibTeX entry (Rao et al., 3 Apr 2026). This arrangement improves accuracy but still allows the model to interleave search, tool use, and metadata revision in ways that produce regressions.

In the two-stage integration, Stage 1 has the model produce a baseline entry using native web search without clibib. Stage 2 is controlled by the client rather than the model: clibib is called using a strict priority order—URL if present, else DOI, else title—and the returned record is accepted only if a title-match gate passes, where lowercased, tokenized titles with stopword filtering must have Jaccard similarity at least 0.3 (Rao et al., 3 Apr 2026). If the title match passes, the model receives both entries in a targeted revision prompt and replaces authors, title, year, venue, volume, number, pages, and DOI with database values, keeping baseline values only for fields the database lacks; if the titles do not match, the baseline is preserved unchanged (Rao et al., 3 Apr 2026).

The study argues that separating search from revision reduces instruction overload and lowers regressions independently of model capability (Rao et al., 3 Apr 2026). This suggests that clibib’s effectiveness is partly architectural: the same retrieval primitive can perform differently depending on whether it is embedded inside a tool-using generation loop or used as a post hoc correction layer.

5. Benchmark, metrics, and empirical performance

clibib is evaluated on a benchmark of 931 papers across Artificial Intelligence, Medicine, Materials Science, and Quantum Computing, stratified into popular, low-citation, and recent tiers (Rao et al., 3 Apr 2026). Three search-enabled models—GPT-5, Claude Sonnet-4.6, and Gemini-3 Flash—generate one BibTeX entry per paper with web search enabled (Rao et al., 3 Apr 2026). The evaluation scores nine fields per entry: entry_type, author, title, year, venue, volume, number, pages, and doi, while entry_key is always marked not applicable because citation keys are arbitrary (Rao et al., 3 Apr 2026).

Baseline performance establishes the problem setting. Overall field-level accuracy is 83.6%, but only 50.9% of entries are fully correct (Rao et al., 3 Apr 2026). Accuracy drops by 27.7 percentage points from popular papers at 92.7% to papers at 65.0%, which the paper interprets as evidence of heavy reliance on parametric memory even when search is available (Rao et al., 3 Apr 2026).

The principal results for clibib are as follows:

Setting	Field-level accuracy	Fully correct entries	Regression rate
Baseline search-enabled LLMs	83.6%	50.9%	—
Single-stage clibib	85.3%	GPT-5: 69.2%; Claude: 70.4%	4.8%
Two-stage clibib	91.5%	78.3%	0.8%

In the single-stage setting, pooled accuracy for GPT-5 and Claude rises from 80.9% to 85.3%, and the fraction of fully correct entries rises from 46.1% to 69.2% for GPT-5 and from 53.0% to 70.4% for Claude (Rao et al., 3 Apr 2026). The largest field gains are number at +19.1 percentage points, volume at +9.0, pages at +7.9, and doi at +7.2 (Rao et al., 3 Apr 2026).

In the two-stage setting, pooled field-level accuracy rises from 83.6% to 91.5%, fully correct entries rise from 50.9% to 78.3%, and regression is held to 0.8% (Rao et al., 3 Apr 2026). Per-model improvements are GPT-5 from 81.4% to 92.5%, Claude from 80.4% to 87.2%, and Gemini from 88.9% to 94.9% (Rao et al., 3 Apr 2026). The largest pooled field gains are number at +22.1 percentage points, doi at +14.9, and pages at +13.0 (Rao et al., 3 Apr 2026). These improvements are concentrated in structured numeric and identifier fields, a pattern consistent with the tool’s reliance on registry data rather than generative reconstruction.

The paper formalizes the principal metrics as

$$A = \frac{\sum_{i=1}^{N} \sum_{f=1}^{F} \mathbf{1}[\text{field}_{i,f}\ \text{is correct}]}{N \times F},$$

for overall field-level accuracy,

$$FCR = \frac{\left|\left\{ i : \sum_{f=1}^{F} \mathbf{1}[\text{field}_{i,f}\ \text{is correct}] = F \right\}\right|}{N},$$

for fully correct entry rate, and

$$\rho = \frac{R}{N_{\text{prev-correct}}}\times 100\%,$$

for regression rate (Rao et al., 3 Apr 2026). The emphasis on regression is important: the paper treats a mitigation layer as successful not merely when it fixes errors, but when it rarely corrupts already correct fields.

6. Error structure, limitations, and operational guidance

The study defines a six-way field-level error taxonomy: Correct, Missing, Fabricated, Partially correct, Substituted, and Not applicable (Rao et al., 3 Apr 2026). It also introduces an error co-occurrence matrix,

$$C_{ab} = P(\text{field } b \text{ wrong} \mid \text{field } a \text{ wrong}) = \frac{|E_a \cap E_b|}{|E_a|},$$

to characterize failure structure (Rao et al., 3 Apr 2026). Two major error modes emerge. The first is wholesale entry substitution, in which identity fields fail together; for example, $$P(\text{title wrong} \mid \text{author wrong}) = 0.71$$, $$P(\text{year wrong} \mid \text{author wrong}) = 0.74$$, $$P(\text{venue wrong} \mid \text{author wrong}) = 0.77$$, $P(\text{doi wrong} \mid \text{year wrong}) = 0.91$, and $P(\text{doi wrong} \mid \text{venue wrong}) = 0.98$ (Rao et al., 3 Apr 2026). The second is isolated field error, especially in formatting and numeric metadata, where co-error rates are lower (Rao et al., 3 Apr 2026).

Among 1,372 entries with at least one error, 65.2% are isolated, 33.7% are mixed, and only 1.0% are wholesale substitutions with at least three substituted fields (Rao et al., 3 Apr 2026). Two-stage clibib fixes 57.9% of isolated-error entries to fully correct but only 14.3% of wholesale substitutions (Rao et al., 3 Apr 2026). This is an important limitation: once the wrong paper has effectively been selected, deterministic revision cannot reconstruct the intended citation if the database path also reflects that wrong target.

Other limitations are coverage and ambiguity. Zotero and CrossRef indexation lags or paywalls can yield not_found, described as approximately 11% of lookups during ground-truth construction, and single-stage success rates vary by domain, from 97.2% in Quantum Computing to 81.8% in AI (Rao et al., 3 Apr 2026). Medicine is specifically noted as being affected by paywalls (Rao et al., 3 Apr 2026). Title-based ambiguity remains a risk when identifiers are absent, even though clibib mitigates it by Jaccard ranking and the two-stage pipeline further gates revision on title similarity (Rao et al., 3 Apr 2026). The paper also notes that normalization rules and synonym tables target English and Latin scripts, and do not extend evaluation to non-English venues (Rao et al., 3 Apr 2026).

Operationally, the recommended pattern is retrieval-centric “search then revise” integration (Rao et al., 3 Apr 2026). The preferred query order is to use stable identifiers surfaced by search telemetry—DOI if available, otherwise publisher or arXiv URL, otherwise title (Rao et al., 3 Apr 2026). The system should route identifiers to Zotero /search, DOI-style URLs to parsed /search, other URLs to Zotero /web, and only then to CrossRef fallback if /search returns no candidates (Rao et al., 3 Apr 2026). Before revision, a title Jaccard threshold should be applied; the evaluated two-stage system used a threshold of at least 0.3 (Rao et al., 3 Apr 2026). The paper further notes that clibib enforces polite behavior with 2 requests per second and normalizes known URL patterns such as arXiv PDF or HTML to the abstract page, and alphaxiv or certain HuggingFace paper links to arXiv (Rao et al., 3 Apr 2026).

In this form, clibib functions as a narrow but effective mitigation layer for structured citation metadata. Its contribution lies less in bibliographic discovery than in enforcing authoritative resolution and in demonstrating that integration architecture—particularly the separation of search from revision—materially affects the reliability of LLM-based scientific writing systems (Rao et al., 3 Apr 2026).