CheckIfExist: AI Citation Verification
- CheckIfExist is an open-source citation verification tool designed to detect and correct AI-generated, fictitious references in academic writing.
- The system uses cascading queries to CrossRef, Semantic Scholar, and OpenAlex with string-similarity and author cross-validation for reliable metadata validation.
- It supports both single and batch processing modes, outputting standardized APA citations and corrected BibTeX records to maintain bibliographic integrity.
Searching arXiv for the specified paper and closely related citation-verification work to ground the article. CheckIfExist is an open-source, web-based reference verification tool for detecting citation hallucinations in AI-assisted academic writing. It is designed to answer a narrow but operationally consequential question: whether a citation actually exists and, if it does, what validated metadata should be returned. The system targets reference hallucination in which LLMs generate plausible-looking but non-existent citations, and it performs real-time validation against CrossRef, Semantic Scholar, and OpenAlex through a cascading architecture with string-similarity scoring and author-level cross-validation (Abbonato, 27 Jan 2026).
1. Problem setting and motivation
CheckIfExist is situated in the broader problem of bibliographic integrity under large-scale use of LLMs. The paper characterizes citation hallucination as the generation of references that appear legitimate at the surface level—plausible author names, journal titles, and years—while corresponding to no real publication. It frames this as an urgent integrity problem because citations support tracing ideas, reproducibility, and bibliometric measurement, yet AI-assisted workflows reduce the cost of producing fabricated references (Abbonato, 27 Jan 2026).
The system is designed for a specific verification task rather than general reference management. Its operative question is: does this citation actually exist, and if so, what is the most reliable validated metadata for it? This focus distinguishes it from tools that primarily organize libraries or format bibliographies. A plausible implication is that CheckIfExist treats citation verification as a first-class validation problem rather than as a by-product of reference storage.
The paper also places the tool in the context of recent concern about hallucinated citations appearing in accepted machine learning papers. That framing matters because it shifts citation checking from editorial hygiene to workflow-integrated integrity control. CheckIfExist is therefore presented as a direct response to AI-generated bibliographic fabrication rather than to ordinary human transcription error alone (Abbonato, 27 Jan 2026).
2. System architecture and verification workflow
The implementation is in React with TypeScript and is intended to run directly in the browser without installation. The workflow begins with preprocessing that removes LaTeX commands such as \vspace, \hspace, \textit, or custom commands, because manuscript-derived references can contain formatting artifacts that interfere with parsing and search (Abbonato, 27 Jan 2026).
After preprocessing, the input is classified as either BibTeX or free-text citation text. BibTeX is decomposed into fields, whereas free-text input is converted into a search query. The verification stage is explicitly cascading and multi-source. CrossRef is queried first and treated as the primary source, with the paper describing it as the most comprehensive DOI-linked scholarly metadata hub, with over 140 million works. The system requests the top three CrossRef candidates and evaluates them for title, author, journal, and year matches. If CrossRef returns nothing, or if the score is weak or issues are detected, the system falls back to Semantic Scholar and then OpenAlex (Abbonato, 27 Jan 2026).
The paper’s rationale for this cascade is that no single source fully covers the literature. Semantic Scholar is described as a secondary source with strong metadata and author disambiguation, while OpenAlex is described as a broad open aggregator drawing from CrossRef, PubMed, ORCID, arXiv, and other repositories. The architecture therefore combines source complementarity with cross-validation of author lists across databases. This suggests that the design is optimized for coverage and contradiction detection rather than for single-database exact lookup.
3. Matching, scoring, and confidence estimation
CheckIfExist relies on normalized Levenshtein similarity for string matching. The paper gives the formula as
where lev(a, b) is Levenshtein distance after converting strings to lowercase and stripping non-alphanumeric characters. This similarity is used for titles, and the paper notes that title similarity above 80% is a key signal (Abbonato, 27 Jan 2026).
Author matching is handled separately. The system extracts family names from source metadata and checks whether those family names are present in the query. It computes author similarity as the proportion of matched authors and also searches for suspicious capitalized tokens that do not correspond to title words, journal names, the year, or any known author family name. These are flagged as potentially fabricated authors. The paper emphasizes this because hallucinated citations can contain invented or chimeric author lists (Abbonato, 27 Jan 2026).
The confidence model is composite and uses both similarity scores and penalties. The text states that if title similarity exceeds 80% but author similarity falls below 90%, confidence is computed using a formula of the form
$\text{confidence} = S_{\text{title} - 0.5 \times (100 - S_{\text{author})$
and for structured input
$\text{confidence} = \frac{S_{\text{title} + S_{\text{author} + S_{\text{journal} + S_{\text{year}{4} + \beta_{\text{ms}$
with $\beta_{\text{ms} \in [0, 10]$ awarded when authors are validated across multiple databases. The paper further specifies penalties for detected problems: title mismatches incur -20, author mismatches -20, journal discrepancies -10 to -20, and each detected fake author -10 to -20. These values are described as empirically calibrated, with author discrepancies weighted more heavily because they are especially diagnostic of AI-generated fabrications (Abbonato, 27 Jan 2026).
The pipeline uses multiple thresholds. The pipeline diagram shows “Score > 70%?” for fallback and “Score > 80%?” for a fully verified result, while the algorithm’s return value indicates existence when bestScore > 50. As a result, the system distinguishes between existence, high-confidence verification, and partial or suspicious matches. That separation is central to the tool’s design: it does not reduce verification to a binary accept/reject rule.
4. Input modes, outputs, and workflow integration
CheckIfExist supports both single-reference verification and batch processing. In “quick check” mode, a citation can be pasted in free-text form, including styles such as APA, MLA, or Chicago, or even informal reference descriptions. In batch mode, the system accepts multiple references either as BibTeX entries or as newline-separated citation lists. Batch processing is sequential and uses an 800 ms rate limit to comply with API usage policies, while the interface provides progressive status updates as each citation is verified (Abbonato, 27 Jan 2026).
The output is not limited to an existence judgment. For verified references, the system generates standardized APA 7th edition citations and valid BibTeX records based on authoritative database metadata rather than on the possibly flawed user input. The batch interface supports export in two forms: downloadable corrected BibTeX files for LaTeX projects or reference managers, and a copy-to-clipboard function for immediate reuse. In this sense, the tool acts as both verifier and metadata correction engine (Abbonato, 27 Jan 2026).
The paper also positions CheckIfExist as complementary to conventional reference managers rather than competitive with them. Zotero, Mendeley, EndNote, and JabRef are acknowledged as strong tools for organizing libraries and generating formatted bibliographies, but the paper stresses that they do not validate authenticity. A plausible implication is that CheckIfExist is intended to sit between AI-assisted drafting and downstream bibliography management, sanitizing references before they are imported into a conventional library.
5. Relation to adjacent citation-verification research
CheckIfExist belongs to a growing line of work on citation verification and hallucination detection, but its scope is narrower than some adjacent systems. The tool focuses on immediate validation against multiple scholarly metadata sources and on returning corrected citation metadata. By contrast, CiteCheck formulates citation verification as a 3-way classification problem—Exact, Minor, and Major—and combines external scholarly retrieval with a structured LLM verifier and calibrated thresholds (Khajavi et al., 26 May 2026).
That distinction is substantive. In CiteCheck, “Minor” means the intended paper exists but one or more bibliographic fields are wrong, whereas “Major” means no sufficiently similar real paper can be found. CheckIfExist also distinguishes between existence, high-confidence verification, and partial or suspicious matches, but the supplied text does not present the same 3-class taxonomy or a retrieval-grounded LLM verification stage. This suggests that CheckIfExist is closer to a practical browser-based validation utility, whereas CiteCheck is a benchmarked hybrid detection framework (Khajavi et al., 26 May 2026).
A second relevant comparison is OpenFactCheck, which provides a unified framework for factual claim verification, LLM factuality evaluation, and checker benchmarking. OpenFactCheck targets claim-level and document-level factual correctness through configurable claim processing, retrieval, and verification pipelines rather than bibliographic authenticity specifically (Wang et al., 2024). CheckIfExist is thus specialized: it verifies citations rather than general factual claims, and its core evidence objects are scholarly metadata records rather than retrieved passages supporting arbitrary propositions.
6. Evaluation status, limitations, and significance
The supplied text does not report a formal benchmark dataset with standard accuracy, precision, or recall numbers for CheckIfExist. The paper instead motivates the tool through design features, operational thresholds, and intended behavior. It states that scoring thresholds and penalty values were empirically calibrated, and it describes response latency qualitatively as “within seconds” for single-reference verification, with batch mode paced at 800 ms between API calls (Abbonato, 27 Jan 2026).
The paper also implies several limitations. The tool depends on database coverage and metadata quality; CrossRef may miss preprints, regional journals, or older works; Semantic Scholar is stronger in some domains but not exhaustive; and OpenAlex has broad coverage but variable metadata quality. It may also struggle with unusual transliterations, incomplete references, nonstandard formatting, or severely corrupted metadata. Partial and ambiguous cases are explicitly surfaced as “Partial Match,” which indicates that the system is designed to expose uncertainty rather than conceal it (Abbonato, 27 Jan 2026).
Its broader significance lies in reframing citation checking as an automated integrity task for the era of generative AI. The core claim is that scholarly workflows need tools operating at the same speed and scale as AI-assisted writing because the cost of producing plausible but false citations has collapsed while manual verification remains expensive. By combining CrossRef, Semantic Scholar, and OpenAlex in a cascading validation architecture and adding author-level cross-validation targeted at fabricated names, CheckIfExist is presented as an attempt to preserve bibliographic integrity without interrupting the writing workflow (Abbonato, 27 Jan 2026).