Reagent-C: Text-Driven Inference Refinement

• The paper introduces Reagent-C, an inference-time approach that refines outputs using targeted textual critiques without updating base model parameters.
• It employs a two-pass mechanism where an initial output is critiqued and then refined, improving performance in agentic reasoning and continual visual learning.
• Empirical results show significant gains (e.g., +3.8% on GAIA and +10.9% on 2Wiki) and demonstrate its practical impact across diverse applications.

Textual-Augmented Refinement (Reagent-C) refers to a class of inference-time techniques in which agentic systems or classifiers are augmented and refined by integrating targeted textual critiques, without updating underlying model parameters. This approach leverages structured feedback—typically natural language critiques—returned by a specialized reward or feedback model, enabling frozen policies or representations to be polished at evaluation time. The method has been instantiated and studied in multi-agent language reasoning workflows (Fan et al., 29 Jan 2026), CLIP-driven continual visual learning (He et al., 3 Aug 2025), and LLM-based prompt optimization (Pandita et al., 5 Jun 2025), with multiple variants demonstrating its flexibility and performance.

1. Definition and Theoretical Underpinnings

Reagent-C is introduced in the context of "Exploring Reasoning Reward Model for Agents" (Fan et al., 29 Jan 2026) as the Textual-Augmented Refinement variant of the broader Reagent family (Reagent-C, -R, -U). The defining features are:

• No parameter updates: The base policy $\pi_\theta$ remains frozen throughout; no RL or gradient optimization is performed.
• Purely inference-time: All refinement is performed at evaluation, not during training.
• Critique-driven: Only the natural language <critique> output from the Agent Reasoning Reward Model (Agent-RRM) is used for refinement.
• Two-pass inference: For a query $q$, an initial output $o^{(1)}$ is generated, critiqued, and then a refined second output $o^{(2)}$ is produced based on $(q, o^{(1)}, \text{critique})$.
• No scalar reward: While Agent-RRM also produces a <score> in $[0,1]$, Reagent-C does not use this for policy optimization.

The essential mathematical structure is:

$$o_i^{(2)} \sim \pi_\theta \left(o \mid q, o_i^{(1)}, c_i\right)$$

where $o_i^{(1)}$ is the initial output and $c_i$ is the <critique> from Agent-RRM.

2. Core Methodology and Algorithmic Workflow

The Reagent-C inference pipeline consists of the following steps for each new query $q$ (Fan et al., 29 Jan 2026):

1. Initial Generation: Sample an initial answer $o^{(1)} \sim \pi_\theta(\cdot|q)$ from the frozen base model.
2. Critique Generation: Apply Agent-RRM to $o^{(1)}$; Agent-RRM returns (i) > : reasoning trace, (ii) <critique>: actionable feedback, (iii) <score>: scalar evaluation (unused).

   1. Prompt Augmentation: Construct an augmented prompt comprising $(q, o^{(1)}, \text{critique})$.

   2. Refined Generation: Sample a refined answer $$o^{(2)} \sim \pi_\theta(\cdot|q, o^{(1)}, \text{critique})$$; return $o^{(2)}$.

   No training or policy update occurs; all improvements arise from the in-context effect of the critique. The critique block is the only modification to the agent’s input. Hyperparameters (e.g., temperature = 0.6, top-p = 0.95) are held fixed at inference time. The backbone in experiments is Qwen3-8B (SFT checkpoint), but the technique is not architecture-specific.

   3. Applications and Domain Instantiations

   3.1. Agentic Reasoning Tasks

   In agentic RL, Reagent-C augments trajectory generation for complex reasoning and tool use. Evaluations on general agent and knowledge-intensive benchmarks reveal that Reagent-C produces consistent performance gains over base policies—e.g., on GAIA (text avg): 25.2% vs. 21.4%, and WebWalkerQA avg: 35.5% vs. 29.0% (Fan et al., 29 Jan 2026). The approach is especially effective where the initial agent reasoning contains subtle or correctable flaws highlighted in the critique.

   3.2. Continual Visual Learning

   Semantic-Enhanced Visual Prototype Refinement (SE-VPR), introduced in "Harnessing Textual Semantic Priors for Knowledge Transfer and Refinement in CLIP-Driven Continual Learning" (He et al., 3 Aug 2025), is another instance of Textual-Augmented Refinement. SE-VPR refines class prototypes in CLIP-based continual learners using relationships derived from text-encoder embeddings. The method involves:

   • Obtaining prompt-augmented textual features $\mathbf{Z}_k$ for each class.
   • Projecting to a latent space with LayerNorm and a learnable matrix, then deriving a normalized affinity matrix $\bar{\mathbf{M}}$ based on Gaussian kernel distances between textual features.
   • Forming semantic-enhanced prototypes:

   $$\hat{\mathbf{c}}_{V,k} = \sum_{j \in {\cal Y}^{1:s}} \bar{\mathbf{M}}_{k,j} \mathbf{c}_{V,j}$$

   • The refined prototypes are then used for classification, yielding gains such as +1.93% on ImageNetA and +1.49% on CIFAR-100 over centroid baselines.

   3.3. Prompt Optimization and Multi-Agent Workflows

   ProRefine (Pandita et al., 5 Jun 2025) operationalizes Textual-Augmented Refinement for prompt design in agentic workflows. The procedure is iterative:

   • Generate a partial output using the current prompt.
   • Solicit textual feedback from an LLM agent, which provides critique and improvement suggestions.
   • Use an optimizer LLM to generate a refined prompt based on this feedback.
   • Repeat for up to $n$ steps or until an automatic verifier determines sufficient improvement.

   ProRefine empirically achieves substantial accuracy improvements across math reasoning datasets (e.g., +18.6% over zero-shot CoT on GSM8K with Llama3.2-1B-instr).

   4. Empirical Performance and Comparative Results

   Across studied benchmarks (Fan et al., 29 Jan 2026), Reagent-C’s empirical profile is characterized by:

   Task/Benchmark	Base (%)	Reagent-C (%)	Margin (%)
   GAIA (text avg)	21.4	25.2	+3.8
   WebWalkerQA (avg)	29.0	35.5	+6.5
   HotpotQA	52.0	61.0	+9.0
   2Wiki	58.0	68.9	+10.9
   MATH500	90.4	93.8	+3.4
   GSM8K	94.6	94.9	+0.3

   Ablations confirm that $o^{(2)}$ (refined) consistently outperforms $o^{(1)}$, indicating the practical value of the textual critique mechanism. However, RL-trained variants (Reagent-R, Reagent-U) yield larger gains, demonstrating the complementary benefit of reward-based supervision.

   In continual visual learning (He et al., 3 Aug 2025), SE-VPR (i.e., Textual-Augmented Refinement) outperforms both classical centroids and prior hybrid methods, fostering semantic consistency and improving discrimination among classes.

   5. Analysis: Strengths, Limitations, and Interpretability

   Strengths:

   • Training-Free Application: Immediately deployable on any frozen policy; no additional training or data required.
   • Generality: Improves results for agentic reasoning, knowledge retrieval, multimodal classification, and prompt design.
   • Interpretability: Critiques explicitly identify logic or tool-use errors, which aids debugging and transparency.
   • Broadly Effective: Gains observed across diverse domains (textual reasoning, vision, prompt optimization).

   Limitations:

   • Bounded Correction: The extent of improvement is limited to what the base model can accommodate in a single second-pass refinement.
   • No Cumulative Learning: Without parameter updates, improvements cannot accumulate; plateauing is observed.
   • Critique Dependency: Effectiveness relies on the accuracy and specificity of Agent-RRM’s (or feedback LLM’s) critique, which itself depends on specialized RL calibration and annotation (e.g., SFT + GRPO training for Agent-RRM).

   6. Extensions, Related Methodologies, and Future Directions

   Multiple research directions arise from current limitations (Fan et al., 29 Jan 2026, Pandita et al., 5 Jun 2025):

   • Critique-augmented fine-tuning: Integrate critique feedback into supervised or RL-based fine-tuning, addressing the static nature of Reagent-C.
   • Multi-pass or Iterative Loops: Extend from a single two-pass regime to multi-round textual refinement for further progressive correction.
   • Hybrid Integration: Bridge Reagent-C with reward-driven approaches (as in Reagent-U), jointly optimizing for scalar performance and critique assimilation.
   • Multi-agent and Knowledge-grounded Feedback: Incorporate multiple feedback agents, retrieval-based grounders, or meta-level aggregators for richer critique aggregation, as conceptualized in possible extensions of ProRefine to full Reagent-C pipelines (Pandita et al., 5 Jun 2025).
   • Prototype Adaptation in Vision: In continual learning, explore dynamic adjustment of $\gamma$ (affinity sharpness), prompt architecture, or explicit regularization schedules for better balancing plasticity and stability (He et al., 3 Aug 2025).

   This suggests that Textual-Augmented Refinement could serve as a modular backbone for scalable, interpretable, and training-free correction in reasoning-centric and multi-agent systems, provided challenges of critique relevance and integration with learning-based optimization are addressed.