Sciwrite-Lint: Manuscript Verification Linter
- Sciwrite-Lint is a local, open-source linter for scientific manuscripts that verifies citation integrity, metadata consistency, and claim support via a non-agentic pipeline.
- It implements a structured verification workflow using deterministic checks and LLM-based methods to assess per-reference reliability and generate a comprehensive SciLint Score.
- Evaluations on diverse research papers show high recall for detecting citation issues while also revealing limitations with PDF-only references and long-document analyses.
Sciwrite-Lint most commonly refers to sciwrite-lint, a local, open-source linter for scientific manuscripts designed for an environment of AI-assisted “science vibe-writing.” It runs entirely on the researcher’s own machine, uses only open-weights models and free public bibliographic APIs, never sends manuscript text to external services, and verifies references, retraction status, metadata consistency, claim support, and one level of bibliography integrity beyond the cited paper itself (Samsonau, 9 Apr 2026). In a broader research sense, “Sciwrite-Lint” also appears as a label for learned or configurable scientific-writing lint infrastructure: one line of work frames it as expert-aligned writing-quality scoring plus edit-based revision, and another as DSL-based compilation of natural-language writing standards into executable lint rules (Chakrabarty et al., 10 Apr 2025, Zhang et al., 8 Feb 2026).
1. Definition, scope, and rationale
The released system is framed as a response to a structural gap in scientific quality assurance. Journal gatekeeping is described as slow, biased, and poor at catching fabricated or misattributed citations even at top venues, while open science provides essentially no systematic checking; in that setting, AI-assisted writing increases both volume and error rate (Samsonau, 9 Apr 2026). The proposed alternative is to measure the paper itself rather than treating venue prestige or authorial integrity as the only filters.
Within that framing, sciwrite-lint is not primarily an authorship detector or a style checker. Its core function is verification: confirming that references exist and are correctly identified, checking retraction status and expressions of concern, comparing metadata against canonical records, downloading and parsing cited papers, asking whether cited papers support the claims made about them, and then checking the cited papers’ own bibliographies for hallucinations, mismatches, and retractions one level deeper (Samsonau, 9 Apr 2026).
The intended usage is correspondingly broad but still manuscript-centered. It can be run by authors before submission, used as a pre-reviewer for journals, or treated as a quality signal for preprints. A plausible implication is that the tool is intended to raise the baseline of structural correctness rather than replace disciplinary judgment about novelty, truth, or significance (Samsonau, 9 Apr 2026).
2. Verification pipeline and operational workflow
sciwrite-lint uses a structured, non-agentic pipeline. The input is a LaTeX project or PDF. Figures are extracted, text is parsed into sections, internal consistency checks run locally, references are verified against public APIs, cited papers are downloaded and parsed when possible, claim support is checked against retrieved passages, and the resulting signals are aggregated into per-reference reliability scores and a report (Samsonau, 9 Apr 2026).
| Stage | Main operation | Representative outputs |
|---|---|---|
| 1 | Figure analysis and internal checks | dangling-cite, numbers-vs-tables, caption-vs-content |
| 2 | Reference identification and verification | tiers T1/T2/T3, metadata mismatches, retraction flags |
| 3 | Download and parse cited papers | section-structured markdown, embeddings, extracted bibliography |
| 4 | Consistency checks on cited papers | warnings/errors on contradictions and reporting |
| 5 | Claim verification and bibliography verification | support scores, citation-purpose labels, bibliography integrity |
| 6 | Aggregation | per-reference reliability scores and overall report |
The first stage combines deterministic checks with local model-based checks. Deterministic checks include dangling-cite, dangling-ref, and unreferenced-figure. Pairwise and full-paper checks include cross-section-consistency, structure-promises, numbers-vs-tables, percentages-sum, sample-size-consistency, arithmetic-consistency, causal-language-audit, abstract-body-alignment, and statistical-reporting. Figure-text checks include caption-vs-content, text-vs-figure, axis-label-consistency, and figure-data-vs-table (Samsonau, 9 Apr 2026).
Reference verification distinguishes entries with structured identifiers from those without them. When DOI, arXiv ID, PMID, ISBN, or LCCN is present, the tool performs batched API queries and cross-ID consistency checks. When no structured identifier is present, a dedicated matching engine performs loose search, retrieves up to 10 candidates per API, and scores candidates using four signals: title similarity, author overlap with name-variant handling, a quadratic year penalty, and venue match as a tiebreaker. The best candidate is accepted at score ; otherwise the reference is classified as T3: not found (Samsonau, 9 Apr 2026).
For references deemed to exist, sciwrite-lint favors a local PDF when available, otherwise attempts to download full text from eight open-access sources. Downloaded PDFs are processed with GROBID into section-structured markdown; full text is embedded; and each cited paper’s bibliography is extracted into structured entries (Samsonau, 9 Apr 2026).
Claim verification is passage-grounded rather than citation-string-grounded. The claim is embedded, relevant sections are retrieved from the cited paper by KNN search over paragraph embeddings, and the LLM classifies the relationship as SUPPORTS, PARTIALLY_SUPPORTS, or NOT_SUPPORTED. If the initial verdict is NOT_SUPPORTED, a context-narrowing step fuzzy-matches the manuscript sentence back to the cited paper and re-verifies within a tighter window (Samsonau, 9 Apr 2026).
3. Reliability scoring, integrity, and the SciLint Score
A distinctive feature of sciwrite-lint is its per-reference reliability score , which aggregates verification tier, retraction status, metadata consistency, cross-ID consistency, cited-paper consistency checks, bibliography integrity, claim support, and citation purpose (Samsonau, 9 Apr 2026).
| Tier or condition | Meaning | Score effect |
|---|---|---|
| T1 | API-verified + full text | base $0.9$ |
| T2 | API-verified, no full text | base $0.7$ |
| T3 | not found in APIs | base $0.3$ |
| Retracted | retracted paper | $0.0$ |
| Expression of Concern | expression of concern |
Metadata mismatches in title, author, year, or venue incur each, and each cross-ID mismatch also incurs . A non-formal document incurs . Consistency penalties on the cited paper are 0 per warning and 1 per error. Bibliography verification adds a hallucination-rate penalty, 2 per bibliography metadata mismatch capped at 3, and 4 per bibliography retraction capped at 5 (Samsonau, 9 Apr 2026).
When both metadata and consistency evidence are available, the score is blended as
6
If the reference is retracted, then 7. For very long documents, with default threshold above approximately 50 pages, cited-paper consistency checks are skipped and the reference is assigned a neutral consistency score of 8 so that citing books is not penalized (Samsonau, 9 Apr 2026).
Above the reference level, the tool defines an experimental SciLint Score:
9
Here $0.9$0 is internal consistency, $0.9$1 is claim verification score, $0.9$2 is per-reference reliability, and $0.9$3 is a citation-purpose weight. Citation-purpose weights are evidence $0.9$4, contrast $0.9$5, method $0.9$6, definition $0.9$7, example $0.9$8, attribution $0.9$9, tool $0.7$0, and context $0.7$1 (Samsonau, 9 Apr 2026).
The rightmost factor, labeled contribution, is explicitly experimental. It operationalizes five frameworks from philosophy of science: empirical content from Popper, progressiveness from Lakatos, explanatory unification from Kitcher, problem-solving effectiveness from Laudan, and test severity from Mayo. In the current implementation the axis weights satisfy $0.7$2 each. If contribution assessment is not run, that factor is set to $0.7$3, so the overall score collapses to the integrity component alone (Samsonau, 9 Apr 2026).
This distinction is central. Integrity verification is described as the core of the tool and is evaluated in the paper; the contribution component is released as experimental code for community development (Samsonau, 9 Apr 2026).
4. Models, data sources, and local security model
The implementation is explicitly local and open-weight. The main LLM is Qwen3 8B served via vLLM, with FP8 quantization and KV-cache optimization. It is used for internal consistency checks, claim verification, citation-purpose classification, and contribution-axis classification. The vision-language component is Qwen3-VL, available in 2B or 8B variants, and used for figure description. The embedding model is Snowflake Arctic Embed M v2.0, used for KNN retrieval of relevant sections during claim verification (Samsonau, 9 Apr 2026).
The tool also depends on public bibliographic and catalog APIs. The paper states that sciwrite-lint uses 10 external APIs; examples include OpenAlex, CrossRef, Semantic Scholar, Open Library, Library of Congress, and Retraction Watch. Retraction Watch data are locally cached. Crucially, only citation metadata such as DOIs, titles, and authors are sent over the network, while manuscript text is never sent externally (Samsonau, 9 Apr 2026).
The implementation stack is Python, distributed as pip install sciwrite-lint, with configuration via .sciwrite-lint.toml. GROBID is used for PDF parsing and TEI XML extraction. The target hardware is a single workstation with one consumer GPU; the example development environment is Windows/WSL2, 64 GB RAM, and an NVIDIA RTX 4000 Ada with 20 GB VRAM (Samsonau, 9 Apr 2026).
The security model is unusually explicit for an academic writing tool. External calls are HTTPS and rate-limited. Manuscripts remain local. The system uses controlled XML parsing, size limits, and redirect validation against DNS-resolved IPs. The LLM is never used to generate URLs or shell commands, and it is invoked only after all network requests complete, making it a terminal node in the dataflow. Prompt-injection mitigations include clear delimiters, anti-injection directives, and strict JSON schemas for outputs (Samsonau, 9 Apr 2026).
A common misconception is that the system is mainly an AI-generation detector. The paper positions publisher-style AI detection as a different class of tool and states that sciwrite-lint focuses on verification: whether evidence and argument are structurally correct (Samsonau, 9 Apr 2026).
5. Evaluation, observed performance, and known failure modes
The main evaluation uses 30 unseen papers drawn from arXiv and bioRxiv across CS/ML, physics, biology, neuroscience, ecology, and medicine. Error injection inserted 68 synthetic errors: fake \cite{} instances and broken \ref{} cross-references. The deterministic checks achieved 38/38 detection for dangling-cite, 29/30 for dangling-ref, and 98.5% recall overall with zero false positives for these deterministic checks in both LaTeX and PDF pipelines (Samsonau, 9 Apr 2026).
In the full-pipeline evaluation, 27 of 30 papers completed fully; 3 failed for technical reasons. SciLint Scores in the current configuration ranged from 0.071 to 0.533, with mean 0.254. The pipeline generated 1,755 findings across 27 papers, approximately 65 findings per paper, dominated by reference-exists and reference-accuracy findings (Samsonau, 9 Apr 2026).
False positive analysis revealed that the main weakness before introduction of the matching engine was reference database checking for PDF-only references without DOIs. Among 379 adjudicated findings from the top 20 findings per paper, Claude Sonnet labeled 14% true positives, 60% false positives, and 26% uncertain. The response was twofold: use structured identifiers extracted from GROBID TEI whenever available, and replace naive title matching with the dedicated matching engine (Samsonau, 9 Apr 2026).
Contribution calibration was conducted separately on 20 open-access papers with 38 ordinal constraints. The standalone contribution component satisfied 30 of 38 constraints, or 79%. LIGO 2016, Graphene 2004, Transformer 2017, and Camerer 2018 ranked in the top tier, while LK-99 2023 and LaCour 2014 ranked in the bottom tier. The paper describes these results as promising but preliminary (Samsonau, 9 Apr 2026).
The limitations are substantial and explicitly acknowledged. A first run on a paper with approximately 50 references can take up to approximately 30 minutes, dominated by claim verification. Vector figures in PDF-only input are not processed as images. Matching without identifiers remains error-prone for unusual formats. Claim-support judgment can fail because retrieval misses the relevant passage or because the LLM misclassifies borderline cases. Long or complex works may be over-penalized unless checks are skipped; the current mitigation is to assign long documents neutral consistency scores when detailed checks are omitted (Samsonau, 9 Apr 2026).
The most important normative caution is that SciLint Score is not presented as a proxy for scientific truth or overall importance. The tool measures structural aspects of integrity and argument quality, and the paper warns against using it in hiring, funding, or gatekeeping without human understanding (Samsonau, 9 Apr 2026).
6. Broader research context, naming, and conceptual extensions
The concrete package sits within a wider cluster of research on scientific-writing linting, though not all of that work describes the same artifact. A 2025 study on writing-quality alignment argues that a “Sciwrite-Lint” system needs three components: a definition and operationalization of writing quality, models that can reliably score and compare texts along that notion of quality, and a workflow that uses those scores at test time to improve drafts. In that framework, expert-based pairwise preferences, dedicated reward models, and best-of-$0.7$4 edit selection provide the core machinery (Chakrabarty et al., 10 Apr 2025). This suggests a complementary conception of Sciwrite-Lint centered on revision quality rather than citation-chain verification.
A second line of work treats scientific-writing linting as a configuration-compilation problem. The DSL-based LintCFG system proposes a tool-agnostic intermediate language, staged LLM parsing, matching, verification, and linter-specific rendering. It explicitly maps that architecture to a hypothetical Sciwrite-Lint for journal or style-guide rules, with PLterms such as Sentence, Paragraph, SectionTitle, FigureCaption, Citation, and PassiveVoice (Zhang et al., 8 Feb 2026). A plausible implication is that scientific-writing lint infrastructure can be split into at least two layers: executable policy compilation and manuscript verification.
A third neighboring result is a semantics-driven lint tool for LLM agent configurations derived from a typed lambda calculus. That work argues that lint rules should arise from operational semantics rather than surface schemas alone, and it proposes joint analysis of declarative configuration and imperative code (Liu, 13 Apr 2026). This suggests a route for future Sciwrite-Lint systems that lint not only manuscripts but also the agentic scientific-writing workflows that generate, revise, or verify them.
The name is also easy to confuse with unrelated acronyms. LINT in clinical trial outcome prediction denotes the Language Interaction Network, a text-first model for approval estimation using BioBERT, a transformer encoder, and GRAM over ICD codes (Gao et al., 2024). LINT in long-tail inference evaluation denotes the Logic-Induced-Long-Tail dataset created by the LINK framework (Li et al., 2023). Neither is the sciwrite-lint manuscript verifier.
Taken together, these works indicate that “Sciwrite-Lint” now spans at least three research meanings: a released local verifier for scientific manuscripts, a broader design pattern for expert-aligned revision and ranking of scientific prose, and a prospective compiler or semantics layer for scientific-writing rules and workflows (Samsonau, 9 Apr 2026, Chakrabarty et al., 10 Apr 2025, Zhang et al., 8 Feb 2026).