PaperJury: Automated Manuscript Governance

• PaperJury is a system for pre-submission hardening that uses a deterministic-versus-semantic split to ensure manuscript integrity.
• It decomposes LaTeX papers into stable, addressable units with a frozen claim spine to preserve core arguments during revision.
• The framework integrates due-process adjudication, risk-proportional patching, and controlled revision to balance issue detection and edit safety.

PaperJury is a closed-loop review-verdict-revise-verify system for the pre-submission hardening of already-written LaTeX computer science papers. It is designed for a setting that differs from drafting assistance because it requires adversarial whole-paper review, explicit no-fix outcomes, and bounded artifact-safe revision. Its central architectural claim is a deterministic-versus-semantic split: deterministic orchestration manages decomposition, a frozen claim spine, a durable ledger, routing, stopping, and exact-once patch application, while semantic agents are limited to bounded review, judgment, and repair (Wang et al., 15 Jun 2026).

1. Task definition and governing thesis

PaperJury addresses manuscripts that are locally fluent but globally unsound. The target failures are manuscript-level problems such as unsupported claims, cross-section contradictions, overstatement of results, and positioning that breaks once the whole paper is considered. The motivating example is a cross-section inconsistency in which one table claims best benchmark performance while a later breakdown table shows a baseline with a higher number. The system therefore treats manuscript improvement as pre-submission hardening rather than generic writing assistance (Wang et al., 15 Jun 2026).

The paper distinguishes this task from adjacent systems. Drafting assistants help write or expand a paper; generic writing assistants improve wording; critique generators stop at comment generation; and judge-centered loops let an LLM critique, judge, revise, and decide when it is done. PaperJury argues that these alternatives lack some combination of durable issue identity across rounds, deterministic routing from critique to adjudication, and manuscript control able to reject invalid concerns or defer author-dependent ones. This motivates the thesis that the “load-bearing safety and completion logic” should reside in deterministic orchestration rather than model discretion (Wang et al., 15 Jun 2026).

The system is organized around a precision-recall-cost trilemma. It aims to find real problems, avoid hallucinated complaints, and do so without the token and runtime expansion associated with exhaustive critique generation. A plausible implication is that PaperJury is best understood as a control architecture for manuscript governance rather than as a stronger reviewer prompt.

2. Deterministic orchestration and persistent manuscript state

The deterministic layer begins with manuscript decomposition. For an input LaTeX manuscript $x$, PaperJury defines a deterministic decomposition $D(x)$ into sections, passages, anchors, and cross-reference targets. This yields stable addressable units before any semantic processing occurs. Review comments and edits are subsequently tied to these anchors rather than to fragile raw spans (Wang et al., 15 Jun 2026).

The second state-bearing object is the frozen claim spine $S$, extracted before revision and described as the manuscript’s “protected semantic backbone.” The function of $S$ is constraint rather than summary: revisions may improve support, wording, and local organization, but should not silently rewrite central claims. Together, $D(x)$ and $S$ establish the system’s basic invariants (Wang et al., 15 Jun 2026).

PaperJury then performs bounded holistic review using a small number of whole-paper reviewers. The reviewer count is $N$, clamped to $[2,4]$, with default $N=3$. Each reviewer reads the full manuscript once and emits evidence-anchored weaknesses plus coverage signals. Review is “holistic” because it spans the entire manuscript, but “bounded” because first-pass critique generation is deliberately limited. The paper contrasts this with exhaustive fan-out baselines that review each unit under multiple lenses; PaperJury instead triggers targeted re-read only when deterministic anti-skim checks indicate weak coverage or poor grounding (Wang et al., 15 Jun 2026).

All discovered concerns are merged into a durable ledger $L$. The ledger stores issue identity, evidence and provenance, verdicts, patch history, and application history. Near-duplicate complaints are consolidated, so the same underlying problem persists as one tracked object across rounds. The paper formalizes the broader issue set as $D(x)$0, with contestability labels $D(x)$1 used for downstream routing. This durable-ledger design is one of PaperJury’s main differences from iterative critique loops that recreate ephemeral issue lists at each pass (Wang et al., 15 Jun 2026).

3. Due-process adjudication and terminal verdicts

After merge, PaperJury applies deterministic routing based on contestability. Mechanical and minor-substantive issues go through a cheaper “polish path,” while contestable substantive-major issues are escalated to a due-process trial. Routing is not left to model discretion. The paper treats this as a central safety property because expensive adjudication is invoked only when a mistaken first-pass decision would be consequential (Wang et al., 15 Jun 2026).

For a routed issue $D(x)$2, the system runs a trial procedure $D(x)$3 and computes a terminal verdict in deterministic code:

$D(x)$4

Here $D(x)$5 is a quorum threshold and $D(x)$6 is a majority threshold. The available terminal outcomes are exactly three:

• invalid-drop
• valid-fixable
• author-required

This three-way verdict space is integral to the system. PaperJury rejects a binary “fix or ignore” scheme because issue validity and machine editability are distinct questions. Some concerns are invalid and should be dropped. Some are valid but should not be auto-edited because they require new experiments, claim-level intent decisions, or author judgment. Only valid-fixable concerns proceed to automated repair (Wang et al., 15 Jun 2026).

The trial itself combines two perspectives: a whole-paper defense that argues against the criticism using global manuscript context, and a decorrelated local-context jury that examines localized evidence with reduced dependence on the original reviewer framing. This is described as a due-process mechanism rather than a free-form model debate. A plausible implication is that PaperJury treats review as an adversarially contestable process, not merely as criticism accumulation.

The direct audit of verdict classes shows that this three-way design is operational rather than decorative. In the evaluation, invalid-drop achieved $D(x)$7, valid-fixable achieved $D(x)$8, and author-required achieved $D(x)$9 (Wang et al., 15 Jun 2026).

4. Bounded revision, guard chains, and exact-once patching

Only issues with $S$0 are allowed to generate candidate patches $S$1. Each patch induces an anchor-bounded diff $S$2, meaning the change is explicitly tied to the deterministic anchor structure created during decomposition. This is meant to limit edit blast radius and prevent local repairs from mutating unrelated claims or neighboring sections (Wang et al., 15 Jun 2026).

Every patch is then passed through a risk-proportional guard chain. The paper formalizes the aggregate result as

$S$3

Risk is tracked with a discrete category $S$4, with labels such as LOW and RISKY. LOW-risk patches take a lighter semantic path; RISKY patches trigger stronger audits. The guard chain includes anchor-bounded diff checks, cross-reference checks, semantic meaning or edit audits when triggered, compile checks, and application journaling (Wang et al., 15 Jun 2026).

The key gating rule for revision is:

$S$5

Thus, only patches attached to valid-fixable issues and passing all guards are applied. This is the system’s principal safety barrier against model-generated overreach (Wang et al., 15 Jun 2026).

Patch application itself is exact-once and journaled. The deterministic orchestrator records application history and supports revert semantics if later checks fail. This is designed to eliminate duplicate edits, stale state, and divergence between ledger state and manuscript state. The same deterministic philosophy governs termination. With $S$6 denoting genuinely new issues in round $S$7 and $S$8 denoting issues closed in round $S$9, the unattended loop stops when the deterministic predicate

$S$0

holds, subject to a five-round cap $S$1 (Wang et al., 15 Jun 2026).

The paper’s evaluation treats clean re-review as a further control measure: after edits, the current manuscript state is re-read afresh rather than continuing from prior model outputs. This reduces contamination from earlier critique framing and is intended to surface new issues introduced by edits.

5. Empirical evaluation and operating characteristics

PaperJury is evaluated in a two-arm expert-review evaluation on 12 held-out LaTeX computer science papers: four each from Vision, NLP, and ML. Arm 1 measures issue discovery quality against expert issue panels using panel-relative precision $S$2, audit-corrected precision $S$3, recall $S$4, and macro-averaged $S$5. Arm 2 audits terminal verdicts, routing, and applied edits, yielding verdict accuracy $S$6, routing accuracy $S$7, edit-safety violation rate $S$8, round count $S$9, and wall-clock runtime $D(x)$0 (Wang et al., 15 Jun 2026).

The main baselines are Forward-Only Rewriter, LLM Critic Only, LLM-as-Judge Review-Revise Loop, and Naive Unbounded Per-(Unit×Lens) Generator. PaperJury’s headline results are:

• $D(x)$1
• $D(x)$2 with audited $D(x)$3 and 95% CI $D(x)$4
• $D(x)$5
• $D(x)$6
• $D(x)$7
• $D(x)$8
• $D(x)$9
• $S$0 rounds, with cap hits on $S$1 papers
• $S$2 hours per paper

The paper also reports an average cost of 6.76 million tokens per paper (Wang et al., 15 Jun 2026).

A concise comparison is given below.

System	Key quality metrics	Safety / cost
PaperJury	$S$3, $S$4, $S$5	$S$6, $S$7 h
LLM Critic Only	$S$8	No verdict/edit audit path
LLM-as-Judge Loop	$S$9, $N$0	$N$1, $N$2, cap hits $N$3
Naive Unbounded Generator	$N$4, $N$5	$N$6 h, 31.4M tokens/paper
Forward-only rewriter	No issue list or verdicts	$N$7, $N$8 h

These results support the paper’s central claim that deterministic orchestration improves the balance among issue quality, adjudication quality, edit safety, and completion behavior relative to judge-centered loops and rewrite-only baselines (Wang et al., 15 Jun 2026).

The ablation study is especially diagnostic. Removing bounded review reduces $N$9 by $[2,4]$0 and increases runtime to 4.81 hours. Removing routing drops verdict accuracy by 0.075 and increases runtime to 3.49 hours. Removing the trial lowers $[2,4]$1 by 0.153 with little change in $[2,4]$2, isolating trial as the main verdict-quality component. Removing the claim spine raises $[2,4]$3 from 0.029 to 0.112. Removing the guard chain raises $[2,4]$4 to 0.181 while reducing runtime to 1.94 hours. This decomposition closely matches the paper’s stated thesis: bounded review mainly affects discovery quality and efficiency, trial mainly affects verdict reliability, and claim-spine plus guard-chain controls mainly affect edit safety (Wang et al., 15 Jun 2026).

The per-domain slices remain relatively stable: Vision, NLP, and ML all achieve $[2,4]$5 between 0.646 and 0.671, $[2,4]$6 between 0.878 and 0.902, and $[2,4]$7 between 0.021 and 0.028. The paper treats these as diagnostic rather than powered comparisons because there are only four papers per family (Wang et al., 15 Jun 2026).

6. Scope, limitations, and relation to adjacent “jury” systems

PaperJury is explicitly bounded to LaTeX computer science manuscripts and to the task of pre-submission hardening. It is not presented as a general writing assistant, a paper drafting system, or a replacement for author judgment. The presence of the author-required verdict class is itself evidence of that boundary: some upheld issues require new experiments, claim-level intent decisions, or author intervention and therefore remain outside automated repair (Wang et al., 15 Jun 2026).

The system’s main practical costs are operational complexity and inference budget. It requires deterministic decomposition, a durable ledger, routing logic, trial machinery, patch guards, compile verification, journaling, and revert semantics. It is also substantially more expensive than a forward-only rewriter, even though it is far cheaper than a naive exhaustive critique generator. The evaluation scope is also limited: 12 papers across three CS subfields, with only partial detail in the paper text on expert annotation protocol and inter-rater reliability (Wang et al., 15 Jun 2026).

Within the broader literature, PaperJury belongs to a growing family of “jury”-oriented evaluation systems, but its object and control philosophy are distinct. SE-Jury is an LLM-as-Ensemble-Judge metric for evaluating generated software artifacts through multiple prompt strategies and dynamic team selection (Zhou et al., 27 May 2025). GrandJury is a collaborative human-evaluation protocol built around dynamic rubrics, time-decayed aggregation, and traceability for settings without absolute ground truth (Cho, 4 Aug 2025). “Vibe Coding on Trial” studies unanimous LLM juries for safety-first text-to-SQL acceptance, emphasizing false-accept control through unanimous committees (Ullah et al., 12 Feb 2026). PaperJury differs from these systems because it is centered on a deterministic orchestration layer that governs manuscript state, adjudication, stopping, and exact-once patch application, while semantic agents are intentionally bounded (Wang et al., 15 Jun 2026).

This suggests a broader classification. SE-Jury and unanimous LLM juries treat “jury” primarily as an evaluation ensemble. GrandJury treats it as a transparent, pluralistic governance protocol for human judgment. PaperJury treats it as due-process review architecture for bounded revision of a structured scientific artifact. In that sense, its most distinctive contribution is not a better reviewer model but a theory of where system authority should reside: semantic agents may read, argue, and draft, but deterministic code holds the ledger, the verdict thresholds, the stopping predicate, and the eraser.