---
title: 'DeepReviewer-14B: LLM Academic Peer Review'
url: https://www.emergentmind.com/topics/deepreviewer-14b-model
type: topic
---

# DeepReviewer-14B: LLM Academic Peer Review

DeepReviewer-14B is a large language model (LLM)-based academic paper review system developed as part of the DeepReview project to address limitations in automated scientific research assessment. Designed to systematically emulate expert reviewers, DeepReviewer-14B integrates structured evaluative stages and evidence-based reasoning, setting new state-of-the-art (SOTA) metrics for LLM-driven peer review [2503.08569].

## 1. Model Architecture

DeepReviewer-14B is derived by full-parameter supervised fine-tuning of Phi-4 14B, an open-source transformer language model. The model preserves the original Phi-4 architectural specifications:

- **Parameter count:** 14 billion (B)
- **Transformer depth:** 40 layers
- **Hidden dimension:** 8192
- **Attention heads:** 32

To accommodate extended input sequences from long manuscripts and complex reasoning traces, rotary positional embeddings are replaced with LongRoPE, yielding a context window of up to 256k tokens in inference and 40k tokens during training. No adapters, LoRA, or external modules are introduced; all backbone parameters are trained end-to-end under DeepSpeed+ZeRO3 optimization for computational efficiency [2503.08569].

## 2. Data and Fine-tuning Regimen

The model is fine-tuned on DeepReview-13K, a curated dataset comprising structured review traces collected from ICLR 2024/2025 OpenReview and arXiv submissions. The dataset includes:

- **Total submissions:** 18,976 (filtered to 13,378 valid samples)
- **Annotations per sample:**
  - Reviewer comments (strengths, weaknesses, queries)
  - Rebuttal dialogues
  - Fine-grained scores: overall (1–10), soundness/presentation/contribution (1–4 each)
  - Meta-review text, final recommendation, binary (accept/reject) decision
- **Split:** 90% training (≈12,091), 10% test (1,286, DeepReview-Bench)
- **Data conversion:** OpenReview and arXiv PDF-to-Markdown via MinerU

The supervised objective uses next-token cross-entropy loss over concatenated review chains $(z_1 \rightarrow z_2 \rightarrow z_3 \rightarrow (s, a))$, where $z_1$ is novelty verification, $z_2$ is multi-dimension review, $z_3$ is reliability verification, $s$ is meta-review text, and $a$ is the accept/reject decision. Fine-tuning is conducted over 23,500 steps with batch size 16, constant learning rate $5 \times 10^{-6}$, AdamW optimizer, and random-truncated 40k-token context windows [2503.08569].

## 3. Multi-Stage Review Framework

DeepReviewer-14B operationalizes academic peer review via three explicit, sequentially chained sub-tasks:

### 3.1. Novelty Verification ($z_1$)
- **Goal:** Verify research originality through retrieval of semantically similar prior work.
- **Process:**
  - Semantic retrieval via Semantic Scholar/OpenScholar.
  - Query/paper embedding: $E(\cdot)$.
  - Similarity: $\mathrm{sim}(q, d) = \langle E(q), E(d) \rangle / (\|E(q)\|\|E(d)\|)$.
  - Retrieval probability: $p(d|q) = \exp(\mathrm{sim}(q,d)/\tau) / \sum_{d'} \exp(\mathrm{sim}(q,d')/\tau)$.
- **Output:** Top-$k$ candidate papers and textual novelty assessment $z_1$.

### 3.2. Multi-Dimension Review ($z_2$)
- **Input:** Review set $R$ and associated rebuttals.
- **Action:** Model (Qwen-2.5-72B-Instruct) reconstructs actionable suggestions and synthesizes multi-axis critiques (soundness, novelty, clarity, ethics) into $z_2$.

### 3.3. Reliability Verification ($z_3$)
- **Self-reflection:** Four-stage verification (methodology, experiment, consistency, overall) using Gemini-2.0-Flash-Thinking.
- **Confidence assignment:** Each comment in $z_2$ is evaluated for evidentiary support and assigned a scalar $c \in [0, 1]$.
- **Meta-review Fusion:** Qwen model integrates $z_1$, $z_2$, $z_3$, and the original meta-review to yield $s$.
- **Implicit evidence fusion:** Weighted sum of comment embeddings:
  $$
  e_{\mathrm{fused}} = \sum_{i} c_i \cdot E(\mathrm{comment}_i)/\sum_{i} c_i
  $$

This explicit structuring is reflected both in fine-tuning supervision and inference pipeline [2503.08569].

## 4. Objectives and Optimization

The loss combines autoregressive cross-entropy terms for each sub-stage:
$$
\mathcal{L} = \mathcal{L}_{z_1} + \mathcal{L}_{z_2} + \mathcal{L}_{z_3} + \mathcal{L}_{s,a}
$$
with all $\lambda_i = 1$ in practice. This structure enforces sequential, evidence-based, and multidimensional reasoning throughout the review generation process. Optimization relies on DeepSpeed+ZeRO3 with the aforementioned hyperparameters for efficient large-scale training [2503.08569].

## 5. Evaluation and Empirical Performance

Evaluation is conducted on DeepReview-Bench (1,286 samples) by four principal tasks:

| Metric                  | Value (DeepReviewer-14B, ICLR 2024) | Notes                                      |
|-------------------------|--------------------------------------|--------------------------------------------|
| Rating MSE              | 1.3137                               | –44.8% vs. CycleReviewer-70B               |
| Decision Accuracy       | 0.6406                               | +1.0 pt vs. baseline                       |
| Spearman Value          | 0.3559                               | +6.04% vs. CycleReviewer-70B               |
| Pairwise Accuracy       | 0.6242                               |                                            |
| Win Rate vs. GPT-o1     | 88.21%                               | LLM-as-judge assessment                    |
| Win Rate vs. DeepSeek-R1| 80.20%                               |                                            |

Further, in test-time scaling (Fast/Standard/Best), the model produces 3k/8k/14.5k output tokens, respectively, with corresponding Spearman correlation increasing from 0.326 to 0.355 (+8.97%). Even the fast mode (3k tokens) surpasses CycleReviewer (6k output tokens). Against adversarial attacks, DeepReviewer-14B's score shifts only by +0.31, compared to +4.26 for Gemini [2503.08569].

## 6. Resources and Reproducibility

All resources are open-source and publicly released:

- **Model weights:** https://huggingface.co/WestlakeNLP/DeepReviewer-7B /14B
- **Code, dataset, demonstration:** http://ai-researcher.net
- **Licensing:** Models, code and dataset adhere to open usage policies, with dataset incorporating CC BY 4.0 OpenReview content.

DeepReviewer-14B is reproducible in full according to the project’s releases, and serves as a new LLM-based benchmark for structured, high-fidelity scientific paper review [2503.08569].

Source: https://www.emergentmind.com/topics/deepreviewer-14b-model