RevAgent: Modular Code Review

• RevAgent Framework is an agent-based system that decomposes code review tasks into specialized agents for improved, context-specific comment generation.
• It utilizes a three-stage process—generation, discrimination, and training—to enhance accuracy and quality in automated review feedback.
• Quantitative and human evaluations show significant performance gains over baselines, with improvements in BLEU, ROUGE-L, METEOR, and semantic similarity metrics.

RevAgent is an issue-oriented, agent-based framework for automated code review comment generation. Designed to address the limitations of monolithic LLM-based and IR/PLM-based models—which lack specialization, suffer from dataset bias, and struggle with context sensitivity—RevAgent decomposes the review comment generation task into specialized agents, each focused on a distinct issue category. Through a meticulously architected multi-agent pipeline, RevAgent improves the relevance, specificity, and quality of generated review comments, establishing a new state-of-the-art in automated code review systems (Li et al., 1 Nov 2025).

1. Motivation and Problem Landscape

Automated code review comment generation aims to reduce human effort while ensuring software quality. Traditional approaches rely on single-model architectures, where the same model is responsible for handling a wide variety of issue types, such as refactorings, bug fixes, testing hints, logging advice, and documentation suggestions. This one-size-fits-all paradigm introduces three core challenges:

• Lack of issue specialization: The model's generalist nature produces superficial or generic comments, especially for complex or underrepresented issue types.
• Dataset bias: Most review corpora are dominated by refactoring issues; models trained on such distributions overfit to dominant patterns and underperform on rarer, higher-impact categories including bugfixes, testing, and logging.
• Context insensitivity: Single-stage pipelines cannot reliably infer which issue category is relevant for a code change (diff), often leading to off-target or uninformative comments.

RevAgent addresses these problems by employing dedicated commentator agents for each issue category and employing a critic agent for selection, thus producing more contextually aligned and informative feedback (Li et al., 1 Nov 2025).

2. Architectural Overview

RevAgent executes a three-stage process: Generation, Discrimination, and Training.

2.1 Generation Stage

• Commentator Agents: Five specialized commentator agents $A_c$, one per category ($$c \in \{\mathrm{Refactoring}, \mathrm{Bugfix}, \mathrm{Testing}, \mathrm{Logging}, \mathrm{Documentation}\}$$), generate candidate review comments $y_c = A_c(x)$ for an input code-diff hunk $x$.
• Prompting: Each agent receives a directive tailored to its expertise (e.g., “If the diff hunk needs to be revised to fix one or more bugs, write a code review.”).
• Formalization: The system collects all candidate pairs $\{ (c, y_c) \mid c \in C \}$.

2.2 Discrimination Stage

• Critic Agent: A dedicated critic agent $D$ receives the diff $x$ and all five candidate comments $\{y_c\}$.
• Selection Mechanism: For each $(x, y_c)$, the critic computes a score $s_c = D(x, y_c)$ and deterministically selects the highest-scoring candidate:

$$c \in \{\mathrm{Refactoring}, \mathrm{Bugfix}, \mathrm{Testing}, \mathrm{Logging}, \mathrm{Documentation}\}$$0

• Execution Model: This two-level agent composition ensures both diversity in candidate generation and rigorous selection.

2.3 Training Stage

• Commentator Fine-Tuning: Each commentator agent is fine-tuned via LoRA on its own category-specific corpus $$c \in \{\mathrm{Refactoring}, \mathrm{Bugfix}, \mathrm{Testing}, \mathrm{Logging}, \mathrm{Documentation}\}$$1, where only the low-rank matrices $$c \in \{\mathrm{Refactoring}, \mathrm{Bugfix}, \mathrm{Testing}, \mathrm{Logging}, \mathrm{Documentation}\}$$2 and $$c \in \{\mathrm{Refactoring}, \mathrm{Bugfix}, \mathrm{Testing}, \mathrm{Logging}, \mathrm{Documentation}\}$$3 are updated in $$c \in \{\mathrm{Refactoring}, \mathrm{Bugfix}, \mathrm{Testing}, \mathrm{Logging}, \mathrm{Documentation}\}$$4.
• Critic Fine-Tuning: For each code-diff $$c \in \{\mathrm{Refactoring}, \mathrm{Bugfix}, \mathrm{Testing}, \mathrm{Logging}, \mathrm{Documentation}\}$$5 and oracle comment/category pair $$c \in \{\mathrm{Refactoring}, \mathrm{Bugfix}, \mathrm{Testing}, \mathrm{Logging}, \mathrm{Documentation}\}$$6, “hard” negative candidates from other categories are retrieved using BM25. The critic receives these as discriminative examples and is fine-tuned to classify the correct category.

Stage	Agents Involved	Key Functions
Generation	5 commentator agents	Per-category candidate review comment generation
Discrimination	1 critic agent	Best issue-comment pair selection
Training	Commentators, Critic	LoRA-based SFT; BM25 retrieval for critic examples

This modular, category-oriented approach contrasts with prior pipelines, advancing specialization and selection efficacy (Li et al., 1 Nov 2025).

3. Evaluation Setup

Dataset

• Curev dataset: Contains 20,000 curated code-diff–comment pairs, spanning five issue categories, with a 75/25 train/test split.

Baselines

• CodeReviewer (CodeT5-based PLM) and CodeReviewer-c (joint category+comment modeling)
• LLaMA-Reviewer (PEFT-fine-tuned LLaMA) and LLaMA-Reviewer-c
• TufanoLLM (prompt-based category prediction followed by comment generation)
• CodeAgent (multi-agent, security review-focused)

Metrics

• BLEU-4: n-gram precision-based quality
• ROUGE-L: sequence-level overlap
• METEOR: semantic and lexicon-aligned quality
• SBERT: semantic similarity score
• Pred. Acc.: issue-category identification accuracy

These criteria enable robust, multi-perspective assessment across both surface-level and semantic content.

4. Quantitative and Qualitative Results

Quantitative Findings

Across four open-source LLM backbones (Llama-3-8B, DeepSeek-Coder-6.7B, Qwen2.5-Coder-7B, Qwen2.5-Coder-14B), RevAgent demonstrates statistically significant improvements versus baseline methods:

• $$c \in \{\mathrm{Refactoring}, \mathrm{Bugfix}, \mathrm{Testing}, \mathrm{Logging}, \mathrm{Documentation}\}$$7BLEU = +12.90 %
• $$c \in \{\mathrm{Refactoring}, \mathrm{Bugfix}, \mathrm{Testing}, \mathrm{Logging}, \mathrm{Documentation}\}$$8ROUGE-L = +10.87 %
• $$c \in \{\mathrm{Refactoring}, \mathrm{Bugfix}, \mathrm{Testing}, \mathrm{Logging}, \mathrm{Documentation}\}$$9METEOR = +6.32 %
• $y_c = A_c(x)$0SBERT = +8.57 %

All measured p-values satisfy $y_c = A_c(x)$1 (e.g., $y_c = A_c(x)$2), supporting the claim of robust gains.

Category-Level Performance

RevAgent substantially increases accuracy for minority and complex categories:

• Bugfix: 21.7 % (vs. <5 % for baselines)
• Testing: 14.2 %
• Logging: 8.3 %
• Documentation: 12.4 %

Human Evaluation

A double-blind study (384 random test diffs, two senior Java developers, Cohen’s $y_c = A_c(x)$3) rates RevAgent’s comments distinctly higher than all baselines:

• Readability: 3.8 ± 0.7 (vs. 3.1 ± 1.0 for LLaMA-Reviewer)
• Accuracy: 3.6 ± 1.0 (vs. 2.8 ± 1.3)
• Category-Matching: 3.5 ± 0.9 (vs. 2.8 ± 0.8)

Judgements highlight greater clarity, precision, and correct identification of issue type.

5. Efficiency Analysis and Design Trade-offs

RevAgent introduces a modest computational overhead:

• Latency: +0.038 s per example compared to LLaMA-Reviewer (approximately 2× slower)
• Token usage: Approximately 2,400 tokens per example (vs. 250 for LLaMA-Reviewer), yet 0.06× those of CodeAgent (the most resource-intensive baseline)
• Relative throughput: 0.46× faster than TufanoLLM

Given the substantial gains in review quality, this trade-off is judged favorable for open-source LLMs.

The framework design allows for specialization via LoRA while maintaining the flexibility to update or expand issue categories independently. The reliance on five primary categories imposes a granularity limit, but the architecture is extensible (Li et al., 1 Nov 2025).

6. Limitations and Future Directions

Manual error analysis attributes low-quality comments primarily to:

1. Business-logic knowledge gaps (22 %)
2. Project-specific coding-standard confusion (48 %)
3. Limited context beyond the code diff (43 %)

Identified limitations and research extensions include:

• Incorporation of repository-level history and documentation to reduce business logic and standardization errors.
• Expansion to a broader taxonomy of issue types for more fine-grained and actionable feedback.
• Scaling to larger or proprietary LLMs, and exploring dynamically composed agent teams at function or class granularity for enhanced context sensitivity.
• A plausible implication is that incorporating global project knowledge or source code history could further ameliorate certain persistent failure modes.

---

RevAgent operationalizes the hypothesis that agent specialization and coordinated selection, rather than model monolithism, are critical for precise, context-aware automated review. The framework's multi-stage pipeline sets a new baseline for automated code review evaluation, particularly on challenging minority and semantically complex issue types (Li et al., 1 Nov 2025).