Iterative Agent Decoding (IAD)

• Iterative Agent Decoding is a method that refines black-box AI outputs through iterative, feedback-driven candidate improvement.
• It integrates techniques like BON sampling alternatives, reinforcement learning, and multi-agent gradient mediation to correct systematic errors.
• Empirical results show 3–10% performance gains in tasks such as Sketch2Code, Text2SQL, and LDPC decoding, enhancing overall inference accuracy.

Iterative Agent Decoding (IAD) encompasses a family of algorithms designed to enhance the inference-time performance of black-box AI agents and structured decoders through iterative, feedback-driven candidate refinement or decision scheduling. IAD is distinguished by the integration of dynamic evaluation and selection mechanisms, typically mediated by an external verifier or reward function that provides guidance across multiple refinement steps. This paradigm is particularly relevant in contexts where model parameters and token-level logits are inaccessible and only API calls are available, or where tasks require the correction of systematic, structured errors not addressed by standard sampling or single-pass decoding (Chakraborty et al., 2 Apr 2025, Habib et al., 2021, Mali et al., 2019).

1. Foundations of Iterative Agent Decoding

IAD fundamentally addresses the inadequacy of single-pass or non-iterative inference methods in agentic tasks characterized by complex, multi-modal, or structured outputs. Denoting the reference black-box policy as $\pi_0(y|x)$ and the (unknown) optimal policy as $\pi^*(y|x)$, the central objective is to minimize the performance gap

$$\Delta(x) = \mathbb{E}_{y\sim\pi^*}\left[R(x, y)\right] - \mathbb{E}_{y\sim\pi_0}\left[R(x, y)\right]$$

where $R(x, y)$ is a scalar reward or verification function specifying task success. Best-of-N (BON) sampling, a widely used black-box inference method, generates $N$ i.i.d. samples and returns the one with maximal reward. However, BON does not leverage iterative feedback and cannot systematically correct recurring errors or efficiently utilize verification signals, motivating the development of IAD (Chakraborty et al., 2 Apr 2025).

2. Algorithmic Structures and Variants

2.1 Verifier-Guided Loop (LLM and Metric Feedback)

The prototypical IAD algorithm for agentic tasks operates in discrete refinement iterations. At each iteration $t$, $K$ new candidate outputs are generated from $\pi_0$ conditioned on the earlier best ($\hat{y}_{t-1}^b$) and worst ($\hat{y}_{t-1}^w$) candidates, along with a textual or prompt-based refinement instruction:

$\pi^*(y|x)$0

All candidates are scored via $\pi^*(y|x)$1, and the best/worst are updated accordingly. The process repeats for $\pi^*(y|x)$2 iterations, returning the best candidate at termination. This refinement loop is mathematically a zeroth-order reward ascent and can adaptively condition the agent's generative process on verifier critiques (Chakraborty et al., 2 Apr 2025).

2.2 Multi-Agent Gradient Mediation (Structured Signal Decoding)

In structured decoding, as exemplified by Sibling Neural Estimator (SNE) architectures for image reconstruction, IAD may involve multiple recurrent agents (RNNs), such as a source estimator and a co-estimator. Each agent receives different spatial context, and during training, gradient-based communication enforces partial alignment while also allowing each agent to specialize on local or global correction. The communication channel is regularized to avoid trivial collapse, and only the source agent is retained at inference (Mali et al., 2019).

2.3 Reinforcement Learning-Based Sequential Scheduling

In the context of LDPC decoding (RELDEC), IAD is formulated as a Markov decision process, where a decoding agent sequentially selects clusters of check nodes to update based on the current hard-decision states and observed rewards. Q-learning is used to optimize the scheduling policy, with meta-RL extensions (AM-RELDEC) enabling rapid adaptation to changing channel conditions. During inference, the agent deterministically schedules updates using the learned Q-table (Habib et al., 2021).

3. Verifier and Feedback Mechanisms

Verifier quality directly determines the upper bound of IAD performance. Empirically effective verifiers include deterministic, task-specific metrics (e.g., Intersection-over-Union (IoU) for layout, execution accuracy for SQL, progress/success rates for Webshop tasks), and LLM-judge architectures that provide both scalar rewards and textual critiques. The LLM-judge can operate in a zero-shot mode, generating targeted feedback that can be explicitly incorporated into subsequent prompts to extract actionable signals from single scalar rewards (Chakraborty et al., 2 Apr 2025).

In image reconstruction, the comparison is rooted in loss functions such as mean-squared error (MSE) over sequential partial outputs, with gradient-mediated feedback between agent networks during training (Mali et al., 2019). For RL-based scheduling, the reward structure encodes the fraction of correctly decoded bits within a cluster, and the agent policy is updated via temporal-difference methods (Habib et al., 2021).

4. Empirical Performance and Comparative Analysis

4.1 Sketch2Code, Text2SQL, and Webshop Tasks

IAD consistently achieves 3–6% absolute performance gains over both BON and state-of-the-art single-turn baselines across Sketch2Code (e.g., +1.95 layout-IoU points for $\pi^*(y|x)$3), Text2SQL (68.05% execution accuracy for IAD+Gemini-1.5-Pro vs. 65.58% for E-SQL+GPT-4o), and Webshop (8–10% absolute increases in success rates) (Chakraborty et al., 2 Apr 2025). The following table summarizes core results:

Task	Baseline (Best)	BON (N=4)	IAD (N=4)	IAD+LLM-Judge
S2C Layout-IoU	Gemini-1.5-Pro: 18.25	24.02	25.97	26.61
Webshop SR %	Gemini-1.5-Pro: 29.3	30.3	38.3	44.7 (GPT-4o)

Gains persist whether verification uses automatic metrics or LLM judges.

4.2 Image Decoding Benchmarks

SNE-based IAD methods outperform single-agent iterative decoders and classical codecs, with up to 1.64 dB PSNR gain over JPEG and 0.37 dB over prior neural baselines, with identical test-time complexity due to co-estimator removal at inference (Mali et al., 2019).

4.3 LDPC Decoding

RELDEC and AM-RELDEC IAD schemes provide 0.2–0.5 dB gains (at fixed FER) relative to flooding, message reduction of 20–40%, and robust adaptation to varying SNR with minimal retraining. The Q-learning framework breaks symmetry in standard update schedules by leveraging dynamic feedback per cluster (Habib et al., 2021).

5. Ablation Studies and Theoretical Insights

Ablations consistently identify adaptive, verifier-guided feedback—not mere stochastic diversity—as the driver of IAD gains. When sampling temperature is reduced to near-deterministic levels, BON's performance plateaus, but IAD continues to accrue improvements from $\pi^*(y|x)$4 to $\pi^*(y|x)$5, evidencing the unique utility of iterative, feedback-driven error correction (Chakraborty et al., 2 Apr 2025). Robustness to mild reward noise and sparsity also differentiates IAD, though extreme levels of either degrade its advantage.

Verifier quality is foundational: high-fidelity, task-aligned verifiers give maximal improvement and facilitate inference-time scaling. With suboptimal verifiers (e.g., noisy LLMs), scaling benefits diminish, especially for IAD compared to BON.

6. Practical Considerations, Limitations, and Future Work

IAD is fundamentally sequential and thus less parallelizable than BON, but achieves higher marginal reward per API call. Design recommendations include maintaining succinct prompts (200–300 tokens), using explicit criticism in feedback where possible, and iterating small candidate batches (2–4) over a modest number of refinements (2–6) (Chakraborty et al., 2 Apr 2025). For structured decoders (e.g., SNE), joint training with auxiliary agents and communication loss provides substantial empirical benefit without increased inference cost (Mali et al., 2019).

Limitations include:

• Tabular RL scaling (in LDPC decoding) is constrained by exponential memory with respect to cluster size; function approximation is proposed for larger block lengths (Habib et al., 2021).
• IAD requires robust reward signals; under >80% reward sparsity or extreme noise, advantages over baseline diminish.

A plausible implication is that as verifier fidelity and structured feedback methods improve, IAD will remain central to inference-time agent optimization under black-box constraints or in domains where systematic and structured error correction is paramount.