Generate–Critique–Refine Loop in AI

• Generate–Critique–Refine Loop is an iterative process where models create candidate outputs, evaluate them against specific criteria, and refine results based on feedback.
• This approach employs distinct actor and critic roles, stateful orchestration, and dynamic stopping rules to systematically enhance output quality.
• Empirical studies show that this feedback-driven refinement framework significantly improves performance in reasoning, code generation, and design tasks.

The Generate–Critique–Refine loop is an iterative pattern in which a model or agent first produces a candidate artifact, then critiques that artifact against task-specific criteria, external evidence, or both, and finally produces a revised candidate conditioned on the critique. Recent work instantiates this pattern for reasoning traces, plans, personas, frontend code, personalized reviews, activity diagrams, educational feedback, design critiques, and even hidden states inside an image editor, with critics that may be self-based, cross-model, algorithmic, visually grounded, or externally supervised (Amini et al., 8 Jun 2026, Tang et al., 24 Jan 2025, Chen, 23 Apr 2026).

1. Formal structure and operational variants

A canonical formulation separates an actor or generator from a critic. In Verbal Process Supervision (VPS), an initial reasoning trajectory is generated as $\tau_0 \sim \pi_\theta(\cdot \mid x)$, a supervisor produces critique $c_r = \mathcal{C}(\tau_r, x)$, and the actor regenerates a revised trajectory as $$\tau_{r+1} \sim \pi_\theta(\cdot \mid x, \tau_r, c_r)$$, repeating up to a round budget $R$ or until a stopping criterion is met (Chen, 23 Apr 2026). RealCritic makes the same pattern explicit in three settings: self-critique, where the same model generates and critiques its own solution; cross-critique, where one model critiques another model’s solution; and iterative critique, where critique and correction are repeated over multiple rounds (Tang et al., 24 Jan 2025).

The stopping rule is a defining part of the loop. In PerGent, the loop “runs for a user-defined number of rounds or stops earlier when the critic indicates that no major issues remain,” and the orchestrator terminates execution when an agent signals completion or a user-defined maximum number of round-robin cycles is reached (Amini et al., 8 Jun 2026). Other systems use fixed caps such as three refinement cycles for frontend code generation, $T=5$ iterations for PerFine, or $k=10$ self-improvement iterations for intrinsic planning self-critique (Sansford et al., 7 Apr 2026, Maram et al., 28 Oct 2025, Bohnet et al., 30 Dec 2025).

The literature also shows that the loop need not always be an outer multi-pass cycle. “Inline Critic Steers Image Editing” embeds critique directly inside a single forward pass of a frozen image editor: a learnable critic token predicts intermediate-layer error structure and then steers later hidden states before denoising finishes (Kang et al., 12 May 2026). By contrast, “Critique Fine-Tuning” treats critique primarily as a training objective and states that “during inference time, the trained model simply generates the response without involving any critique or refinement process” (Wang et al., 29 Jan 2025).

2. Components, orchestration, and control flow

A recurring design pattern is explicit orchestration. PerGent uses a centralized orchestrator, a generator agent, and a critic agent. The orchestrator controls turn order with a round-robin policy, stores the shared conversation state in orchestrator.history, and passes each agent a prompt built from role, instructions, actionList, memory, and history. The generator produces an initial persona when history is empty and otherwise refines; the critic evaluates the latest persona and emits either feedback or an approval declaration (Amini et al., 8 Jun 2026). This makes the loop stateful rather than stateless prompt chaining.

Other systems add control modules before or around critique. DeCRIM introduces a decomposer that rewrites a multi-constraint instruction into an enumerated list of explicit constraints and then lets a critic decide which constraints failed before refinement is invoked (Ferraz et al., 2024). DCR inserts a Detect stage before critique so that only errorful sentences are critiqued and refined, rather than forcing the critique model to both verify and explain everything (Wadhwa et al., 2024). REFINE decomposes feedback into components such as current-state diagnosis, task-level next steps, strategy-level guidance, self-regulated learning support, and praise, and then regenerates only the failing components identified by a human-aligned LLM judge (Fawzi et al., 31 Mar 2026).

Orchestration also determines how harmful refinements are contained. PerFine uses a knockout strategy that compares the current and previous drafts and retains the stronger one across rounds (Maram et al., 28 Oct 2025). The frontend critic-in-the-loop framework maintains a best-so-far candidate across the refinement trajectory rather than assuming the final cycle is optimal (Sansford et al., 7 Apr 2026). LADEX restarts generation if a candidate still fails critique after five refinement iterations, and it reports that no input required more than one restart (Khamsepour et al., 3 Sep 2025). These variants show that refinement is often treated as a search process over a trajectory of candidates rather than a monotonic improvement operator.

3. Forms of critique and sources of supervision

The content of the critique varies sharply across domains. In math reasoning, DeepCritic trains a critic to perform deliberate step-wise critiques with multi-perspective verifications and in-depth critiques of initial critiques for each reasoning step, returning the first erroneous step or $-1$ if all steps are correct (Yang et al., 1 May 2025). AutoMathCritique similarly targets step-level natural-language feedback and the first error location, and then uses that critique both at test time and in self-improvement training (Xi et al., 2024). VPS requires the supervisor to label each step as correct — do not revise, partially correct — describe the issue, or incorrect — give a targeted correction, and to emit CONVERGED when everything is correct (Chen, 23 Apr 2026).

Several papers distinguish critique from scalar scoring. RealCritic argues that critique should be evaluated through a closed-loop critique-and-correction framework rather than by judging critique text in isolation (Tang et al., 24 Jan 2025). CLoud reward models implement a candidate response $\rightarrow$ generated critique $\rightarrow$ scalar reward pipeline, showing that critique can mediate scoring even when explicit refinement is replaced by selection (Ankner et al., 2024). CFT goes further and trains directly on supervision of the form $([x;y], c)$, where the model critiques a query–noisy-response pair instead of imitating a final answer (Wang et al., 29 Jan 2025).

Grounding sources also differ. PerGent conditions both generator and critic on external resources and target persona description, where the resources include 25 interview transcripts, 28 surveys, and 47 job postings across ten roles (Amini et al., 8 Jun 2026). In intrinsic planning self-critique, the critic is scaffolded by domain rules: for each action it must take the action and its preconditions from the domain definition, verify whether the preconditions are met, and apply the action to provide the resulting state (Bohnet et al., 30 Dec 2025). PerFine grounds critique in a retrieved user profile and asks for structured feedback on Tone Consistency, Vocabulary Match, Sentence Structure, and Topic Relevance (Maram et al., 28 Oct 2025). Frontend code generation uses a vision-LLM as a visual critic over rendered screenshots, followed by a code critic that translates visual observations into actionable code-oriented recommendations (Sansford et al., 7 Apr 2026). UI design critique generation likewise iteratively refines both design comments and the corresponding bounding boxes that ground each comment in the screenshot (Duan et al., 2024).

A broader pattern is that effective critiques are usually more than verdicts. DeepCritic’s long-form critiques, DCR’s error-span-plus-fix feedback, PerFine’s profile-grounded JSON feedback, and CLoud’s critique-before-score architecture all treat critique as a semantically rich object rather than a scalar label (Yang et al., 1 May 2025, Wadhwa et al., 2024, Maram et al., 28 Oct 2025, Ankner et al., 2024).

4. Empirical evidence across domains

Reported results consistently show that critique-conditioned refinement can improve quality, but the magnitude depends on the task, the critic, and the grounding signal.

Domain	Reported effect	Citation
Industrial persona authoring	96.90% approval, 3.10% edit, 0% remove, 0% add	(Amini et al., 8 Jun 2026)
Symbolic planning (Blocksworld 3–5)	49.8±4.0% to 85.5±2.8%; 89.3±2.5% with self-consistency	(Bohnet et al., 30 Dec 2025)
Frontend code generation	up to 17.8% increase over three refinement cycles	(Sansford et al., 7 Apr 2026)
Reasoning with VPS	94.9% on GPQA Diamond at $c_r = \mathcal{C}(\tau_r, x)$0; weak-actor rescue from 11.7–26.7% to 63.3–90.0% on AIME 2025	(Chen, 23 Apr 2026)
Activity diagram extraction	up to 86.37% correctness and 88.56% completeness, with fewer than five LLM calls on average	(Khamsepour et al., 3 Sep 2025)
Inline image editing	GEdit-Bench 7.89, +9.4 gain on RISEBench over the same backbone, and KRIS-Bench 81.92	(Kang et al., 12 May 2026)

Other domains show the same pattern. DeCRIM improves Mistral’s performance by 7.3% on RealInstruct and 8.0% on IFEval, and with strong feedback open-source LLMs with DeCRIM can outperform GPT-4 on both benchmarks (Ferraz et al., 2024). DeepCritic raises Qwen2.5-7B-Instruct from 74.00 to 77.20 on MATH500 and raises Qwen2.5-72B-Instruct from 77.00 to 82.00 under critique-based refinement (Yang et al., 1 May 2025). PerFine reports GEval gains of +7–13% over PGraphRAG, with steady improvements over 3–5 refinement iterations and gains that scale with critic size (Maram et al., 28 Oct 2025). REFINE reports that judge-guided regeneration significantly improves feedback quality, while its interactive agent produces responses comparable to a state-of-the-art closed-source model (Fawzi et al., 31 Mar 2026). CLoud reward models improve RewardBench pairwise preference classification accuracy by 4.65 points for Llama-3-8B and 5.84 points for Llama-3-70B, and also improve Best-of-N selection on ArenaHard (Ankner et al., 2024). In multimodal UI critique, iterative visual prompting reduces the gap from human performance by 50% for one rating metric (Duan et al., 2024).

5. Evaluation paradigms, misconceptions, and failure modes

A central methodological dispute concerns how critique quality should be measured. RealCritic argues that open-loop evaluation is misleading because critique is open-ended, and it reports that about 30% of low-quality critiques are erroneously classified as high-quality by verdict-style evaluation; among critiques judged low-quality by humans, 26.0% still had verdicts matching the ground truth (Tang et al., 24 Jan 2025). The paper therefore scores critique by whether it produces a better corrected answer. This position is echoed by systems that evaluate the whole loop against expert edits, reference artifacts, or downstream task correctness rather than stylistic plausibility alone.

A second misconception is that high approval implies completeness. PerGent explicitly warns against this: although PerGent reached 96.90% approval in in-situ expert review, its comparison with pre-LLM expert personas showed average full preservation 0.424 and partial preservation 0.678, meaning a nontrivial amount of expert-authored content was still missing (Amini et al., 8 Jun 2026). The paper’s “Lesson 2” states that strong expert approval of a rich draft should not be mistaken for evidence of completeness.

A third misconception is that self-critique or extra rounds automatically help. RealCritic finds that, in self-critique, classical LLMs may even underperform relative to their baseline capabilities, and only o1-mini shows positive average self-critique improvement among the tested models (Tang et al., 24 Jan 2025). CFT reaches the same caution from the training side: its critique-trained models “currently lack the ability to perform self-critique,” and direct inference outperforms their tested self-critique variants (Wang et al., 29 Jan 2025). VPS reports non-monotonic round scaling across $c_r = \mathcal{C}(\tau_r, x)$1 and attributes degradation to over-correction, compounding errors, and critique drift (Chen, 23 Apr 2026). Frontend code generation shows monotonic mean gains but also emphasizes performance regressions and the diminishing returns of the critique; this is exactly why the algorithm keeps the best candidate found across the trajectory (Sansford et al., 7 Apr 2026).

Failure modes are strongly domain dependent. VPS reports that performance degrades when decisive errors are not linguistically expressible, especially in code synthesis, and therefore motivates hybrid verbal–executable methods (Chen, 23 Apr 2026). RealCritic shows that specialized STEM multiple-choice tasks such as GPQA and MMLU-STEM are particularly dangerous for critique-refine loops, with large C→I degradations in some settings (Tang et al., 24 Jan 2025). By contrast, intrinsic self-critique becomes much more effective in symbolic planning because the prompt can force the model to verify action preconditions and state transitions under explicit domain rules (Bohnet et al., 30 Dec 2025).

6. Recurrent design patterns and research directions

Across the literature, several design patterns recur. One is role separation: actor and critic are distinct in PerGent, VPS, CGI, DeepCritic, AutoMathCritique, and many evaluation setups, even when the same base model family is reused (Amini et al., 8 Jun 2026, Chen, 23 Apr 2026, Yang et al., 20 Mar 2025, Yang et al., 1 May 2025, Xi et al., 2024). Another is statefulness: successful industrial and agentic systems preserve conversation history, prior drafts, critiques, or failed attempts rather than issuing isolated prompts (Amini et al., 8 Jun 2026, Bohnet et al., 30 Dec 2025). A third is schema or component awareness: personas use a fixed four-section template, REFINE judges pedagogical components separately, LADEX separates structural and alignment constraints, and PerFine critiques exactly four personalization dimensions (Amini et al., 8 Jun 2026, Fawzi et al., 31 Mar 2026, Khamsepour et al., 3 Sep 2025, Maram et al., 28 Oct 2025).

The literature also repeatedly narrows the critic’s job before asking it to refine. DCR adds Detect so critique focuses only on flagged sentences (Wadhwa et al., 2024). DeCRIM adds Decompose so critique is constraint-wise rather than holistic (Ferraz et al., 2024). LADEX replaces LLM structural critique with deterministic checks whenever the property is formally machine-verifiable, and this alone improves correctness by 17.81% and completeness by 13.24% over LLM-only checks (Khamsepour et al., 3 Sep 2025). REFINE preserves passing feedback components and regenerates only the failing ones (Fawzi et al., 31 Mar 2026). A plausible implication is that critique becomes substantially more useful when verification, localization, and revision scope are disentangled.

Another recurring theme is that critique ability is itself a trainable capability. CFT shows that learning to critique noisy responses can outperform answer imitation by 4–10% on math benchmarks (Wang et al., 29 Jan 2025). DeepCritic uses deliberate step-wise critiques followed by reinforcement learning to produce stronger feedback for later refinement (Yang et al., 1 May 2025). AutoMathCritique turns incorrect trajectories into critique-conditioned self-improvement data, improving exploration efficiency and solution diversity (Xi et al., 2024). CGI similarly trains the actor not only to solve tasks but also to utilize critiques, and its ablations show that removing critique-conditioned training data causes the largest performance drop (Yang et al., 20 Mar 2025). CLoud adds a cautionary systems lesson: the downstream scorer must be trained on the self-generated critique distribution it will actually consume, because off-policy oracle critiques substantially degrade performance (Ankner et al., 2024).

Research directions in the current papers are unusually concrete. VPS argues for hybrid verbal–executable supervision when errors are not fully observable in language (Chen, 23 Apr 2026). Frontend code generation points to richer process distillation rather than sequence-level distillation of only the final refined output (Sansford et al., 7 Apr 2026). PerFine highlights dynamic stopping criteria and dynamic retrieval updates across iterations (Maram et al., 28 Oct 2025). LADEX suggests domain-specific support such as RAG for semantic alignment critique in specialized industrial texts (Khamsepour et al., 3 Sep 2025). Inline Critic points toward a broader class of systems in which critique happens within a forward pass rather than after an external draft is complete (Kang et al., 12 May 2026).

Taken together, the current literature supports a precise characterization: effective Generate–Critique–Refine systems are typically stateful, schema-constrained, evidence-aware, and evaluated in closed loop; they work best when the critic is stronger, better grounded, or more formal than the actor on the failure modes that matter most.