Iterative Assessment Algorithm (IAA)

Updated 13 November 2025

IAA is a family of methods that iteratively refines scores or candidate outputs by aggregating both intrinsic attributes and relational feedback.
It is applied in network-based fraud detection and sequence generation, achieving notable performance gains in AUC and output accuracy.
The algorithm uses smoothing, domain-specific heuristics, and verifier-guided feedback to ensure near-monotonic improvements across iterations.

The Iterative Assessment Algorithm (IAA) is a broad family of inference-time procedures that incrementally refine candidates or risk scores based on both individual attributes and relational or extrinsic feedback. The IAA framework has emerged independently in two principal contexts: (1) relational credit assignment in entity networks for expert-driven fraud detection, specifically automobile insurance fraud, and (2) iterative refinement of sequence generation in black-box LLM decoding. The unifying principle is the propagation and aggregation of evaluative signals over multiple rounds, leading to monotonic or near-monotonic improvement in ranking or solution quality.

1. Formal Definition and Core Framework

IAA comprises iterative loops that update the score or representation of each entity or candidate using both intrinsic factors (internal attributes) and extrinsic relational or model-derived feedback. In both domains—graph-based prediction and sequence generation—the IAA operates over either nodes in a network or candidate solution sequences, with each round using a combination of current state, new evaluations, and domain- or task-specific heuristics.

The update step takes the general form: $s^{k}(en_i) = \alpha s^{k-1}(en_i) + (1-\alpha) AM_{\cdot}^{mean}(en_i)$ where $s^k(en_i)$ is the score at iteration $k$ , $\alpha$ is an inertia/smoothing parameter, and $AM^{mean}_{\cdot}$ is a mean-normalized aggregate function over neighbors or candidates, as context-appropriate. Aggregation functions can incorporate individual entity attributes, relational edge-types, and external verifier signals.

In sequence generation (Chakraborty et al., 2 Apr 2025), IAA—termed Iterative Agent Decoding (IAD)—maintains a pool of candidates and iteratively samples, evaluates, and refines them via verifier-guided feedback.

2. Applications in Network-Based Fraud Detection

The foundational application of IAA is in the detection and ranking of fraud within automobile accident claim networks, as described by expert systems leveraging social network structures (Šubelj et al., 2011). In this regime:

Vertices $V$ : represent entities (e.g., participants, accident buckets).
Edges $E$ : encode relationships (co-involvement, driver/passenger roles).
Scores $s^k$ : refined suspicion probabilities per entity.
Update Mechanism: Scores are updated based on a weighted combination of neighbors' prior scores and domain-specific risk factors (intrinsic and relational), normalized by network degree.

Key equations (from (Šubelj et al., 2011)):

Raw model (no attributes): $AM_{raw}(en_i) = \sum_{e=\{v_i,v_j\}\in E(v_i)} s^{k-1}(en_j)$

Basic model (incorporating risk factors): $AM_{bas}(en_i) = F_{int}(en_i) \sum_{e=\{v_i,v_j\}\in E(v_i)} F_{rel}(\ell(e)) s^{k-1}(en_j)$

Degree normalized for robustness: $AM^{mean}_{\cdot}(en_i) = \frac{\overline{d} + d(v_i)}{2} \frac{AM_{\cdot}(en_i)}{d(v_i)}$

Expert-driven settings use a small, hand-tuned set of risk factors $f^k_{int}$ and $F_{rel}$ to encode domain knowledge about suspicious behaviors.

3. Iterative Agent Decoding for Sequence Generation

In black-box AI agent inference, IAA enables dynamic improvement over conventional sampling methods by using a verifier to guide candidate refinement at runtime (Chakraborty et al., 2 Apr 2025). This is particularly effective in environments where model parameters are fixed and only API access is available.

High-level schema:

Generator ( $\pi_0$ ): Produces candidate outputs from a context or prompt.
Verifier ( $R(x, y)$ ): Assigns scalar reward to each candidate.
Selector: Chooses top-k and (optionally) bottom-k for further feedback.
Feedback Loop: Incorporates best/worst candidates into subsequent context to elicit improved outputs.

Pseudocode (informal summary):

for t in range(1, T+1):
    candidates = [generator(context_{t-1}) for _ in range(N)]
    scores = [verifier(x, y) for y in candidates]
    if best_{t-1} exists:
        candidates.append(best_{t-1})
        scores.append(verifier(x, best_{t-1}))
    best_t = top_k(candidates)
    worst_t = bottom_k(candidates)
    context_t = x + InstructionTemplate(best_t, worst_t)
    if stopping_criterion_met:
        break
return best_t

Monotonicity and early-stopping are achieved by always preserving the best-so-far candidate across rounds.

4. Empirical Performance and Model Variants

Fraud Detection (network context):

Comparative AUC performance on labeled Slovenian automobile insurance data (Šubelj et al., 2011):

Model Variant	AUC
IAA $_{raw}$	0.8872
IAA $_{bas}$	0.9145
IAA $_{raw}^{mean}$	0.8942
IAA $_{bas}^{mean}$	0.9228
Best Eigenvector Centrality	~0.858

IAA $_{bas}^{mean}$ , with expert-tuned risk factors and dynamic stopping at max component-diameter, outperforms all classical centrality and unsupervised baselines, achieving recall $\approx0.89$ , specificity $\approx0.87$ , precision $\approx0.65$ , and F1 $\approx0.75$ .

Sequence Generation (IAD context):

IAA-inspired schemes yield absolute gains of 3–6% over Best-of-N (BON) sampling on structured tasks such as Sketch2Code and Text2SQL, and 8–10% on Webshop (Chakraborty et al., 2 Apr 2025). For example:

Sketch2Code: IAD (N=4, T=2) achieves layout-sim ≈25.97 (vs BON's 24.0).
Text2SQL (BIRD): IAD + Gemini-1.5-Pro achieves $\sim68\%$ execution accuracy (vs SOTA 63%).
Webshop: IAD with GPT-4o improves success rate from ≈40.3% (base) to ≈44.7%.

Ablation studies demonstrate that improvements are verifier-guided rather than attributable to sampling diversity alone; robustness is maintained under modest reward sparsity and moderate verifier noise.

5. Algorithmic Complexity and Parameterization

IAA presents favorable computational characteristics in both principal applications:

Network-based IAA: Per iteration complexity is $O(|E_c| + |V_c|)$ ; total iterations set to $\max\{\mathrm{diam}(c), \lceil \overline{d}_G \rceil\}$ for efficiency and to prevent over-propagation.
Sequence generation IAD: Each round incurs $N$ generator/verifier queries, total cost roughly $T \cdot N$ . Early-stopping is employed to prevent wastage when no further gain is observed.

Essential tuning parameters:

$\alpha$ (smoothing/inertia, typically 0.75): balances prior scores and new aggregate assessments.
Intrinsic/relational risk factors: calibrated on labeled data.
Number of iterations ( $T$ ), candidates per round ( $N$ ), top/bottom-k selection ( $k$ ), and minimum improvement threshold ( $\epsilon$ ).

6. Practical Recommendations for Deployment

Risk scoring: Normalize outputs post-update to ensure comparability and convergence. In fraud detection, rank participants within each connected subgraph to present interpretable risk lists to investigators.
Sequence refinement: Always inject both best and worst candidates into feedback to enable zeroth-order improvement signals. Limit feedback context size (200–300 tokens) to avoid overfitting.
Verifier selection: Use low-cost or automated verifiers when feasible; for code, integration with unit tests or executable oracles is advantageous.
Robustness: With noisy or sparse supervision signals, ensemble verifiers and/or increase candidate pool size.

7. Illustrative Example: Fraud Network Propagation

A minimal network with three participants $\{A,B,C\}$ distributed across two collision buckets (A–X–B–Y–C) demonstrates the effect of the iterative aggregation process. Initial scores are uniform; after successive mean-normalized updates and smoothing, participant $B$ —with links to both $A$ and $C$ —accumulates greater suspicion, reflecting its central topological and relational position. This process, repeated across subgraphs, yields rank-ordered lists for investigator review, with empirical results confirming alignment with ground-truth fraud labels.

The Iterative Assessment Algorithm, in both network and sequence generation settings, supports principled, inference-time propagation of evaluative signals to improve risk assignment or solution quality, leveraging relational, intrinsic, and verifier-derived cues. Its modular structure and empirical efficacy render it broadly adaptable to other structured or relational domains.