---
title: 'Sciwrite-Lint: Manuscript Verification Linter'
url: https://www.emergentmind.com/topics/sciwrite-lint
type: topic
---

# Sciwrite-Lint: Manuscript Verification Linter

Sciwrite-Lint most commonly refers to **sciwrite-lint**, a local, open-source linter for scientific manuscripts designed for an environment of AI-assisted “science vibe-writing.” It runs entirely on the researcher’s own machine, uses only open-weights models and free public bibliographic APIs, never sends manuscript text to external services, and verifies references, retraction status, metadata consistency, claim support, and one level of bibliography integrity beyond the cited paper itself [2604.08501]. In a broader research sense, “Sciwrite-Lint” also appears as a label for learned or configurable scientific-writing lint infrastructure: one line of work frames it as expert-aligned writing-quality scoring plus edit-based revision, and another as DSL-based compilation of natural-language writing standards into executable lint rules [2504.07532][2602.07783].

## 1. Definition, scope, and rationale

The released system is framed as a response to a structural gap in scientific quality assurance. Journal gatekeeping is described as slow, biased, and poor at catching fabricated or misattributed citations even at top venues, while open science provides essentially no systematic checking; in that setting, AI-assisted writing increases both volume and error rate [2604.08501]. The proposed alternative is to **measure the paper itself** rather than treating venue prestige or authorial integrity as the only filters.

Within that framing, sciwrite-lint is not primarily an authorship detector or a style checker. Its core function is **verification**: confirming that references exist and are correctly identified, checking retraction status and expressions of concern, comparing metadata against canonical records, downloading and parsing cited papers, asking whether cited papers support the claims made about them, and then checking the cited papers’ own bibliographies for hallucinations, mismatches, and retractions one level deeper [2604.08501].

The intended usage is correspondingly broad but still manuscript-centered. It can be run by authors before submission, used as a pre-reviewer for journals, or treated as a quality signal for preprints. A plausible implication is that the tool is intended to raise the baseline of structural correctness rather than replace disciplinary judgment about novelty, truth, or significance [2604.08501].

## 2. Verification pipeline and operational workflow

sciwrite-lint uses a structured, **non-agentic pipeline**. The input is a LaTeX project or PDF. Figures are extracted, text is parsed into sections, internal consistency checks run locally, references are verified against public APIs, cited papers are downloaded and parsed when possible, claim support is checked against retrieved passages, and the resulting signals are aggregated into per-reference reliability scores and a report [2604.08501].

| Stage | Main operation | Representative outputs |
|---|---|---|
| 1 | Figure analysis and internal checks | `dangling-cite`, `numbers-vs-tables`, `caption-vs-content` |
| 2 | Reference identification and verification | tiers T1/T2/T3, metadata mismatches, retraction flags |
| 3 | Download and parse cited papers | section-structured markdown, embeddings, extracted bibliography |
| 4 | Consistency checks on cited papers | warnings/errors on contradictions and reporting |
| 5 | Claim verification and bibliography verification | support scores, citation-purpose labels, bibliography integrity |
| 6 | Aggregation | per-reference reliability scores and overall report |

The first stage combines deterministic checks with local model-based checks. Deterministic checks include `dangling-cite`, `dangling-ref`, and `unreferenced-figure`. Pairwise and full-paper checks include `cross-section-consistency`, `structure-promises`, `numbers-vs-tables`, `percentages-sum`, `sample-size-consistency`, `arithmetic-consistency`, `causal-language-audit`, `abstract-body-alignment`, and `statistical-reporting`. Figure-text checks include `caption-vs-content`, `text-vs-figure`, `axis-label-consistency`, and `figure-data-vs-table` [2604.08501].

Reference verification distinguishes entries with structured identifiers from those without them. When DOI, arXiv ID, PMID, ISBN, or LCCN is present, the tool performs batched API queries and cross-ID consistency checks. When no structured identifier is present, a dedicated matching engine performs loose search, retrieves up to 10 candidates per API, and scores candidates using four signals: title similarity, author overlap with name-variant handling, a quadratic year penalty, and venue match as a tiebreaker. The best candidate is accepted at score $\ge 0.70$; otherwise the reference is classified as **T3: not found** [2604.08501].

For references deemed to exist, sciwrite-lint favors a local PDF when available, otherwise attempts to download full text from eight open-access sources. Downloaded PDFs are processed with GROBID into section-structured markdown; full text is embedded; and each cited paper’s bibliography is extracted into structured entries [2604.08501].

Claim verification is passage-grounded rather than citation-string-grounded. The claim is embedded, relevant sections are retrieved from the cited paper by KNN search over paragraph embeddings, and the LLM classifies the relationship as `SUPPORTS`, `PARTIALLY_SUPPORTS`, or `NOT_SUPPORTED`. If the initial verdict is `NOT_SUPPORTED`, a context-narrowing step fuzzy-matches the manuscript sentence back to the cited paper and re-verifies within a tighter window [2604.08501].

## 3. Reliability scoring, integrity, and the SciLint Score

A distinctive feature of sciwrite-lint is its **per-reference reliability score** $R(r_i)$, which aggregates verification tier, retraction status, metadata consistency, cross-ID consistency, cited-paper consistency checks, bibliography integrity, claim support, and citation purpose [2604.08501].

| Tier or condition | Meaning | Score effect |
|---|---|---|
| T1 | API-verified + full text | base $0.9$ |
| T2 | API-verified, no full text | base $0.7$ |
| T3 | not found in APIs | base $0.3$ |
| Retracted | retracted paper | $0.0$ |
| Expression of Concern | expression of concern | $\times 0.3$ |

Metadata mismatches in title, author, year, or venue incur $-0.1$ each, and each cross-ID mismatch also incurs $-0.1$. A non-formal document incurs $-0.2$. Consistency penalties on the cited paper are $-0.05$ per warning and $-0.10$ per error. Bibliography verification adds a hallucination-rate penalty, $-0.05$ per bibliography metadata mismatch capped at $-0.30$, and $-0.15$ per bibliography retraction capped at $-0.30$ [2604.08501].

When both metadata and consistency evidence are available, the score is blended as
$$
R(r_i)=\operatorname{clamp}_{[0,1]}\!\bigl(0.4\,M(r_i)+0.6\,C(r_i)\bigr).
$$
If the reference is retracted, then $R(r_i)=0$. For very long documents, with default threshold above approximately 50 pages, cited-paper consistency checks are skipped and the reference is assigned a neutral consistency score of $1.0$ so that citing books is not penalized [2604.08501].

Above the reference level, the tool defines an experimental **SciLint Score**:
$$
S(p)=I(p)\times \operatorname{wmean}_{i}\bigl[w_i\,V(p,r_i)\,R(r_i)\bigr]\times \sum_{a\in\mathcal{A}}\beta_a\,C_a(p).
$$
Here $I(p)$ is internal consistency, $V(p,r_i)$ is claim verification score, $R(r_i)$ is per-reference reliability, and $w_i$ is a citation-purpose weight. Citation-purpose weights are evidence $1.0$, contrast $0.9$, method $0.8$, definition $0.7$, example $0.6$, attribution $0.5$, tool $0.4$, and context $0.2$ [2604.08501].

The rightmost factor, labeled **contribution**, is explicitly experimental. It operationalizes five frameworks from philosophy of science: empirical content from Popper, progressiveness from Lakatos, explanatory unification from Kitcher, problem-solving effectiveness from Laudan, and test severity from Mayo. In the current implementation the axis weights satisfy $\beta_a = 0.2$ each. If contribution assessment is not run, that factor is set to $1.0$, so the overall score collapses to the integrity component alone [2604.08501].

This distinction is central. Integrity verification is described as the core of the tool and is evaluated in the paper; the contribution component is released as experimental code for community development [2604.08501].

## 4. Models, data sources, and local security model

The implementation is explicitly local and open-weight. The main LLM is **Qwen3 8B** served via vLLM, with FP8 quantization and KV-cache optimization. It is used for internal consistency checks, claim verification, citation-purpose classification, and contribution-axis classification. The vision-language component is **Qwen3-VL**, available in 2B or 8B variants, and used for figure description. The embedding model is **Snowflake Arctic Embed M v2.0**, used for KNN retrieval of relevant sections during claim verification [2604.08501].

The tool also depends on public bibliographic and catalog APIs. The paper states that sciwrite-lint uses **10 external APIs**; examples include OpenAlex, CrossRef, Semantic Scholar, Open Library, Library of Congress, and Retraction Watch. Retraction Watch data are locally cached. Crucially, only citation metadata such as DOIs, titles, and authors are sent over the network, while manuscript text is never sent externally [2604.08501].

The implementation stack is Python, distributed as `pip install sciwrite-lint`, with configuration via `.sciwrite-lint.toml`. GROBID is used for PDF parsing and TEI XML extraction. The target hardware is a single workstation with one consumer GPU; the example development environment is Windows/WSL2, 64 GB RAM, and an NVIDIA RTX 4000 Ada with 20 GB VRAM [2604.08501].

The security model is unusually explicit for an academic writing tool. External calls are HTTPS and rate-limited. Manuscripts remain local. The system uses controlled XML parsing, size limits, and redirect validation against DNS-resolved IPs. The LLM is never used to generate URLs or shell commands, and it is invoked only after all network requests complete, making it a terminal node in the dataflow. Prompt-injection mitigations include clear delimiters, anti-injection directives, and strict JSON schemas for outputs [2604.08501].

A common misconception is that the system is mainly an AI-generation detector. The paper positions publisher-style AI detection as a different class of tool and states that sciwrite-lint focuses on **verification**: whether evidence and argument are structurally correct [2604.08501].

## 5. Evaluation, observed performance, and known failure modes

The main evaluation uses **30 unseen papers** drawn from arXiv and bioRxiv across CS/ML, physics, biology, neuroscience, ecology, and medicine. Error injection inserted 68 synthetic errors: fake `\cite{}` instances and broken `\ref{}` cross-references. The deterministic checks achieved 38/38 detection for `dangling-cite`, 29/30 for `dangling-ref`, and 98.5% recall overall with zero false positives for these deterministic checks in both LaTeX and PDF pipelines [2604.08501].

In the full-pipeline evaluation, 27 of 30 papers completed fully; 3 failed for technical reasons. SciLint Scores in the current configuration ranged from 0.071 to 0.533, with mean 0.254. The pipeline generated 1,755 findings across 27 papers, approximately 65 findings per paper, dominated by `reference-exists` and `reference-accuracy` findings [2604.08501].

False positive analysis revealed that the main weakness before introduction of the matching engine was reference database checking for PDF-only references without DOIs. Among 379 adjudicated findings from the top 20 findings per paper, Claude Sonnet labeled 14% true positives, 60% false positives, and 26% uncertain. The response was twofold: use structured identifiers extracted from GROBID TEI whenever available, and replace naive title matching with the dedicated matching engine [2604.08501].

Contribution calibration was conducted separately on 20 open-access papers with 38 ordinal constraints. The standalone contribution component satisfied 30 of 38 constraints, or 79%. LIGO 2016, Graphene 2004, Transformer 2017, and Camerer 2018 ranked in the top tier, while LK-99 2023 and LaCour 2014 ranked in the bottom tier. The paper describes these results as promising but preliminary [2604.08501].

The limitations are substantial and explicitly acknowledged. A first run on a paper with approximately 50 references can take up to approximately 30 minutes, dominated by claim verification. Vector figures in PDF-only input are not processed as images. Matching without identifiers remains error-prone for unusual formats. Claim-support judgment can fail because retrieval misses the relevant passage or because the LLM misclassifies borderline cases. Long or complex works may be over-penalized unless checks are skipped; the current mitigation is to assign long documents neutral consistency scores when detailed checks are omitted [2604.08501].

The most important normative caution is that SciLint Score is **not** presented as a proxy for scientific truth or overall importance. The tool measures structural aspects of integrity and argument quality, and the paper warns against using it in hiring, funding, or gatekeeping without human understanding [2604.08501].

## 6. Broader research context, naming, and conceptual extensions

The concrete package sits within a wider cluster of research on scientific-writing linting, though not all of that work describes the same artifact. A 2025 study on writing-quality alignment argues that a “Sciwrite-Lint” system needs three components: a definition and operationalization of writing quality, models that can reliably score and compare texts along that notion of quality, and a workflow that uses those scores at test time to improve drafts. In that framework, expert-based pairwise preferences, dedicated reward models, and best-of-$N$ edit selection provide the core machinery [2504.07532]. This suggests a complementary conception of Sciwrite-Lint centered on **revision quality** rather than citation-chain verification.

A second line of work treats scientific-writing linting as a **configuration-compilation** problem. The DSL-based LintCFG system proposes a tool-agnostic intermediate language, staged LLM parsing, matching, verification, and linter-specific rendering. It explicitly maps that architecture to a hypothetical Sciwrite-Lint for journal or style-guide rules, with PLterms such as `Sentence`, `Paragraph`, `SectionTitle`, `FigureCaption`, `Citation`, and `PassiveVoice` [2602.07783]. A plausible implication is that scientific-writing lint infrastructure can be split into at least two layers: executable policy compilation and manuscript verification.

A third neighboring result is a semantics-driven lint tool for LLM agent configurations derived from a typed lambda calculus. That work argues that lint rules should arise from operational semantics rather than surface schemas alone, and it proposes joint analysis of declarative configuration and imperative code [2604.11767]. This suggests a route for future Sciwrite-Lint systems that lint not only manuscripts but also the agentic scientific-writing workflows that generate, revise, or verify them.

The name is also easy to confuse with unrelated acronyms. **LINT** in clinical trial outcome prediction denotes the **Language Interaction Network**, a text-first model for approval estimation using BioBERT, a transformer encoder, and GRAM over ICD codes [2405.06662]. **LINT** in long-tail inference evaluation denotes the **Logic-Induced-Long-Tail** dataset created by the LINK framework [2311.07237]. Neither is the sciwrite-lint manuscript verifier.

Taken together, these works indicate that “Sciwrite-Lint” now spans at least three research meanings: a released local verifier for scientific manuscripts, a broader design pattern for expert-aligned revision and ranking of scientific prose, and a prospective compiler or semantics layer for scientific-writing rules and workflows [2604.08501][2504.07532][2602.07783].

Source: https://www.emergentmind.com/topics/sciwrite-lint