Code Review Agents in AI-Driven Reviews

• Code Review Agents are AI-powered systems that automate code review by analyzing code changes, detecting defects, and providing structured, actionable feedback.
• They employ architectural patterns such as role decomposition, issue-oriented specialization, and repository-aware agency to integrate smoothly with pull-request workflows.
• Benchmark studies reveal that while CRAs can enhance defect detection and reduce review latency, human oversight remains crucial to mitigate noise and ensure contextual accuracy.

Code Review Agents (CRAs) are automated or AI-based reviewers that inspect code changes, pull requests, or repository state and produce review feedback such as inline comments, vulnerability reports, suggested patches, or structured review artifacts. In recent literature, the term covers both bot-based pull-request reviewers and more explicitly agentic systems that can understand code context, plan review actions, interact with development environments, and maintain memory across steps. The resulting research area spans automated review comment generation, secure pre-commit review, repository-aware reasoning, workflow orchestration across the pull-request lifecycle, and human-supervised review pipelines (Zhong et al., 16 Mar 2026, Charoenwet et al., 27 Jan 2026, Kamalı et al., 17 May 2026).

1. Definition and historical trajectory

Code review has long been treated as a primary quality gate in software development; one recent position paper explicitly traces this lineage to Fagan’s formalisation of code inspection in 1976 and argues that review has historically served four overlapping goals: defect detection, style and standards enforcement, knowledge transfer, and team awareness (Monperrus, 11 Jun 2026). CRA research inherits those goals but re-specifies the reviewer as a software agent rather than a purely human participant.

The scope of the term varies by paper. In large-scale GitHub studies, CRAs are AI review bots such as GitHub Copilot, CodeRabbit, Devin, Gemini Code Assist, and Claude Code that participate in inline review conversations much like human reviewers do (Zhong et al., 16 Mar 2026). In secure review work, a CRA is an autonomous code review agent operating in a diff-centric CLI environment with repository navigation, explicit tool use, and security-focused memory (Charoenwet et al., 27 Jan 2026). Vision papers further broaden the notion into an end-to-end, AI-powered, multi-agent, human-in-the-loop workflow spanning PR Creation, PR Augmentation, Reviewer Selection, AI-Assisted Code Review, and PR Retrospective (Kamalı et al., 17 May 2026).

The historical transition from review assistance to more agentic review is visible in the literature. RAID, a refactoring-aware diff tool, did not itself act as an autonomous reviewer, but it augmented GitHub diffs with refactoring metadata and reduced the median number of lines to be reviewed from 14.5 to 2 for move refactorings and from 113 to 55 for extractions, thereby lowering cognitive effort in refactoring-heavy review (Brito et al., 2021). Later work such as CodeAgent made the reviewers themselves multi-agent LLM systems, explicitly modelling review as collaborative interaction rather than one-shot generation (Tang et al., 2024).

2. Architectural patterns

A recurring CRA design pattern is role decomposition. CodeAgent organises review into four phases—Basic Info Sync, Code Review, Code Alignment, and Document—and assigns six characters, namely User, CEO, CPO, CTO, Reviewer, and Coder. Its distinctive mechanism is QA-Checker, a supervisory agent that iteratively repairs prompts when an answer is not sufficiently aligned with the original review question. On vulnerability analysis, the full system reported 449 confirmed vulnerabilities with a Rate$_{cr}$ of 92.96%, whereas CodeAgent$_{w/o}$ reported 413 confirmed vulnerabilities with a Rate$_{cr}$ of 73.23%, indicating that conversational supervision materially affected precision (Tang et al., 2024).

A second pattern is issue-oriented specialisation. RevAgent decomposes review comment generation into a Generation Stage, where five category-specific commentator agents draft candidate comments; a Discrimination Stage, where a critic agent selects the most appropriate issue-comment pair; and a Training Stage, where commentators and critic are fine-tuned on curated category-specific data. Its taxonomy includes Refactoring, Bugfix, Testing, Logging, and Documentation, while Others is treated as noise and excluded from training. Averaged across four LLM backbones, RevAgent improved over the best baseline by 12.90\% on BLEU, 10.87\% on ROUGE-L, 6.32\% on METEOR, and 8.57\% on SBERT, and human evaluation rated it highest on readability, accuracy, and category-matching (Li et al., 1 Nov 2025).

A third pattern is repository-aware, tool-using agency. AgenticSCR formalises secure review as

$$\text{AgenticSCR} = \langle \mathcal{E}, \mathcal{A}, \mathcal{G}, \mathcal{M}, \mathcal{T}, \Pi \rangle,$$

where $\mathcal{E}$ is the CLI/repository environment, $\mathcal{A}=\{a_d,a_v\}$ are two specialist subagents, $\mathcal{G}$ are the review goals, $\mathcal{M}$ is the memory system, $\mathcal{T}$ is the toolset, and $\Pi$ is the coordination policy. Its workflow is explicitly detector $_{w/o}$0 validator: the detector localises suspicious lines, generates a security review comment, and predicts a CWE-based type, while the validator checks whether the finding is a real security issue and filters false positives. The design is grounded by security-focused semantic memory containing SAST/CodeQL rules for the detector and a CWE tree derived from the CWE-1000 taxonomy for the validator (Charoenwet et al., 27 Jan 2026).

More programmatic visions extend CRA architecture from a single review event to the full pull-request lifecycle. The five-stage framework proposed in “Rethinking Code Review in the Age of AI” specifies a PR Creation Agent, PR Augmentation Agent, Reviewer Suggestion Tool, PR Review Agent, and retrospective summarisation machinery, while preserving humans at explicit quality gates such as PR author verification, reviewer assignment, approval of automated findings, and final merge authority (Kamalı et al., 17 May 2026). This lifecycle view shifts the unit of design from “comment generation” to “review process orchestration.”

3. Benchmarks and evaluation logic

CRA evaluation has diversified because review is not well captured by single-reference text similarity. CRScore explicitly frames code review as a one-to-many problem and replaces reference matching with a reference-free metric grounded in claims, implications, and code smells extracted from the diff (Naik et al., 2024). CR-Bench instead asks whether a review comment is a BUG_HIT, a VALID_SUGGESTION, or NOISE and makes signal-to-noise ratio a first-class metric (Pereira et al., 10 Mar 2026). c-CRAB converts human review intent into executable tests, so the final oracle is behavioural rather than textual (Zhang et al., 24 Mar 2026). SCRBench targets repository-aware, line-level immature vulnerabilities at the pre-commit stage (Charoenwet et al., 27 Jan 2026).

Benchmark or metric	Evaluation unit	Key reported property
CRScore (Naik et al., 2024)	Review comment quality	Reference-free; Spearman correlation 0.5431 with human relevance
CR-Bench / CR-Evaluator (Pereira et al., 10 Mar 2026)	Review comments on real PRs	BUG_HIT / VALID_SUGGESTION / NOISE; SNR
c-CRAB (Zhang et al., 24 Mar 2026)	PR review plus review-guided revision	Test-based; 184 PRs and 234 validated comments/tests
SCRBench (Charoenwet et al., 27 Jan 2026)	Pre-commit secure review	144 code changes, 107 CVEs, 33 CWE types

CRScore measures three dimensions. Conciseness is the fraction of review sentences aligned above threshold with some pseudo-reference, comprehensiveness is the fraction of pseudo-references covered by the review, and relevance is the harmonic mean of the two. Using pseudo-references grounded in LLM-generated claims and static-analysis smells, CRScore Rel reached a Spearman correlation of 0.5431 with human relevance Likert scores, substantially above BLEU, ROUGE-L, chrF, BERTScore, and other reference-based metrics (Naik et al., 2024).

CR-Bench and CR-Evaluator make a different point: defect-detection recall alone is insufficient because false positives directly affect developer productivity. They therefore report Recall, Precision, F1, Usefulness Rate, and

$_{w/o}$1

This framing exposes a hidden frontier between issue resolution and spurious findings (Pereira et al., 10 Mar 2026).

c-CRAB shifts evaluation to executable consequences. For each instance $_{w/o}$2, with validated tests $_{w/o}$3 and a patch $_{w/o}$4 produced after a coding agent is guided by a CRA’s reviews, the score is

$_{w/o}$5

The benchmark was curated from 671 PRs and 1,313 comments, then filtered to 184 PRs and 234 validated comments/tests spanning 67 repositories (Zhang et al., 24 Mar 2026).

4. Empirical behaviour in pull-request workflows

Large-scale observational evidence consistently shows that present-day CRAs behave differently from human reviewers. In 278,790 inline code review conversations across 300 open-source GitHub projects, AI-agent reviews were narrower and more verbose, with over 95% of AI comments concentrated in Code Improvement and Defect Detection. Human reviewers supplied broader feedback, including Understanding, Testing, Knowledge Transfer, and Social comments. Understanding feedback alone accounted for 31% of comments when humans reviewed human-written code and 17% when humans reviewed agent-generated code, and human reviewers exchanged 11.8% more rounds when reviewing AI-generated code than human-written code. AI suggestions were adopted at 16.6%, compared with 56.5% for human suggestions, and when adopted they produced larger increases in code complexity and code size than human suggestions (Zhong et al., 16 Mar 2026).

Reviewer composition also correlates with outcome quality. In the AIDev-based study of CRA-only versus human-only reviewing, the main comparison was restricted to Commented PRs because that is where CRA-only reviewing appears. Among those PRs, CRA-only review achieved a 45.20% merge rate, whereas human-only review achieved 68.37%; CRA-only PRs also had a 34.88% closed rate versus 21.60% for human-only PRs. The association between reviewer type and PR outcome was statistically significant with $_{w/o}$6, $_{w/o}$7, $_{w/o}$8. In a signal analysis of 98 closed CRA-only PRs, 59 PRs (60.2%) fell into the 0–30% signal range, and 12 of 13 CRAs had average signal ratios below 60% (Chowdhury et al., 3 Apr 2026).

Benchmark studies reveal the same precision–noise tension under controlled conditions. On CR-Bench-verified, a single-shot GPT-5.2 reviewer reached Recall 27.01%, Precision 3.56%, F1 6.30%, Usefulness 83.63%, and SNR 5.11, while Reflexion increased Recall to 32.76% but lowered Usefulness to 47.72% and SNR to 1.95. GPT-5-mini under Reflexion reached Recall 27.59% but SNR 0.91, illustrating that aggressive bug-finding can degrade signal integrity (Pereira et al., 10 Mar 2026). c-CRAB yields a related conclusion at the review-to-repair level: Claude Code achieved 32.1% overall, Devin 24.8%, PR-Agent 23.1%, and Codex 20.1%, while the union across all four tools solved 97 out of 234 tests, about 41.5%, leaving a substantial gap between current CRA output and human review intent as encoded by executable tests (Zhang et al., 24 Mar 2026).

5. Security-critical and domain-specific review agents

Security review is one of the most technically ambitious CRA subfields. AgenticSCR targets immature vulnerabilities in pre-commit review, where changes are small, context is incomplete, and latency matters. On SCRBench, AgenticSCR generated 211 comments and achieved 17.5% overall correctness (LRT), whereas a static LLM-based secure reviewer generated 652 comments but only 6.9% LRT. CodeQL achieved 0.6% LRT, Semgrep 3.7%, and Snyk 0.9%. The paper’s headline claim is that AgenticSCR achieves at least 153% relatively higher percentage of correct code review comments than the static LLM baseline. Performance varied by high-level vulnerability category: Injection 23.4%, Authorization 21.4%, Information 6.2%, Resource 16.7%, and Control 0.0%. The ablation study showed a Base agent at 7.3% LRT, Base + SAST Rules at 13.0%, and Base + SAST Rules + CWE at 17.5%, isolating the contribution of structured security memory (Charoenwet et al., 27 Jan 2026).

Compiler optimisation review shows a different but equally specialised CRA design. Archer, described as the first automated agentic code review tool for compiler optimisations, constrains review from both ends: obligations guide analysis and a deterministic validation guard admits only findings backed by executable evidence. Evaluated on 70 open PRs and 328 closed PRs in LLVM from the last two months, Archer found that 21% of open PRs and 11% of closed PRs were buggy. Across the real-world dataset it reported 51 semantic bugs, including 34 miscompilations and 17 crashes. On the regression dataset, direct LLM review found no bugs, mini-SWE-agent found 1 bug, and Optimuzz found 3 bugs, all also covered by Archer, underscoring the value of domain-specific obligations and proof-or-execution-backed validation (Ni et al., 2 Jul 2026).

At the same time, secure CRA systems themselves create new attack surfaces. CoTDeceptor targets CoT-enhanced LLM detectors and vulnerability-review agents by exploiting their reasoning chains and semantic abstraction processes. Its headline result is evasion of 14 out of 15 vulnerability categories, compared to only 2 bypassed by prior methods. The paper argues that reasoning traces can leak decision boundaries, turning interpretability into a double-edged property for security review pipelines embedded in CI/CD or repository auditing (Li et al., 24 Dec 2025).

6. Governance, reviewability, and competing visions

Recent work increasingly treats CRA effectiveness as a governance problem rather than only a model-capability problem. “Software Delegation Contracts” defines delegated coding work as a tuple $_{w/o}$9 consisting of Task, Authority, Returned work package, and Acceptance context. In a controlled pilot study with 64 agent executions and 192 reviews, explicit contracts did not improve objective task outcomes—every run passed hidden acceptance checks, with zero scope violations—but they improved reviewability: evidence sufficiency improved in 22 of 30 paired comparisons and worsened in none, with a mean gain of +0.83 on a 5-point scale ($_{cr}$0, Cliff’s delta = 0.66). The overhead was +13% agent tokens and +38% wall-clock time, indicating that review-facing evidence can be purchased at measurable cost (Schmalbach, 14 Jun 2026).

Governance also appears in triage models for agent-authored PRs. “Early-Stage Prediction of Review Effort in AI-Generated Pull Requests” analysed 33,707 agent-authored PRs from 2,807 repositories and identified a two-regime behavioural pattern: 28.3% of all PRs were instant merges, defined as less than 1 minute from creation to merge, while the remainder entered iterative review cycles with phenomena such as agentic ghosting. The proposed Circuit Breaker triage model used only static structural features at creation time; a LightGBM model reached AUC 0.9571 with 95% CI [0.955, 0.962] on a temporal split, and at a 20% review budget intercepted 69% of total review effort. This reframes CRA deployment as review-effort governance rather than purely automated judgement (Minh et al., 2 Jan 2026).

The field is also divided over whether CRAs should augment or replace human reviewers. “The End of Code Review: Coding Agents Supersede Human Inspection” argues that coding agents have crossed a threshold at which traditional human code review is no longer a necessary component of a software quality pipeline, and proposes structured agent sign-off in place of a human approval checkbox for routine changes (Monperrus, 11 Jun 2026). Empirical work points in the opposite direction: human–AI collaboration studies conclude that human oversight remains critical for contextual feedback, validation of suggestion quality, and final confirmatory judgement, while PR-outcome studies conclude that CRAs should augment rather than replace human reviewers because standalone CRA feedback is often low-signal and associated with higher abandonment (Zhong et al., 16 Mar 2026, Chowdhury et al., 3 Apr 2026). The lifecycle framework in “Rethinking Code Review in the Age of AI” therefore retains humans at key decision points throughout PR Creation, PR Augmentation, Reviewer Selection, AI-Assisted Code Review, and PR Retrospective, treating HITL quality gates as safeguards against hallucination, context loss, cascading errors, bias, privacy risk, and automation bias (Kamalı et al., 17 May 2026).

Taken together, the literature presents CRAs not as a single tool category but as a family of review systems with different operational assumptions: some act as inline conversational reviewers, some as specialist comment generators, some as secure pre-commit agents, and some as full pull-request workflow orchestrators. The strongest empirical findings so far are mixed. CRAs can scale defect screening, exploit repository context, use tools, and integrate structured knowledge; yet they also remain vulnerable to noise, mislocalisation, context loss, low adoption, and adversarial manipulation. Current evidence therefore supports a technically ambitious but operationally constrained view: CRA research is advancing rapidly, but robust deployment still depends on benchmark design, evidence-grounded evaluation, domain-specific scaffolding, and carefully placed human oversight.