GRASP: Graph-Based Argument Ranking

• GRASP is a deterministic, graph-based ranking method that converts local pairwise attack judgments into a global argument ordering.
• It employs a damped propagation operator with explicit two-hop defense integration to compute continuous strength scores for each argument.
• GRASP ensures reproducibility and transparency in debate evaluation by quantifying structural sufficiency without claiming factual or persuasive authority.

GRASP, short for Gradual Ranking with Attacks and Support Propagation, is a deterministic graph-based argument ranking method proposed as an alternative to opaque end-to-end “LLM-as-a-Judge” verdicts in debate evaluation. Its central premise is that a debate should not be collapsed into a single holistic score when the interaction structure among arguments can be represented explicitly and aggregated transparently. In GRASP, a judge model first produces many local judgments about whether one argument attacks another, and those judgments are then aggregated into a global ranking by a fixed propagation operator over an interaction graph. The method is presented as a way to obtain rankings that are deterministic, transparent, and auditable, and as a graded account of structural sufficiency rather than persuasion, factuality, or rhetorical appeal (Misra et al., 18 May 2026).

1. Motivation and target concept

GRASP is motivated by an empirical and methodological critique of holistic LLM judging. In the paper’s framing, a common evaluation practice is to present a whole debate to a model and ask for a final ranking or verdict. The authors argue that this procedure suffers from substantial inter-model disagreement because it compresses a complex argumentative interaction structure into a single opaque score. By contrast, local interaction judgments—in particular, pairwise attack judgments—are treated as more reproducible. GRASP is the mechanism that converts those local signals into a global ordering.

The concept that GRASP is designed to measure is structural sufficiency. This is a graph-relative, defense-aware notion of argument robustness. It is “structural” because it depends only on the explicit interaction graph, and it is “sufficiency” because it asks whether an argument withstands the attacks that appear in that graph. The paper contrasts this with broader notions of global sufficiency, which may require addressing anticipated, implicit, or hypothetical objections. A Boolean precursor is given as

$$\mathrm{SS}(a;\mathcal{G}) \;\Longleftrightarrow\; \forall b \in A,\; (b,a) \in R^{-} \Rightarrow \exists c \in A \text{ such that } (c,b) \in R^{-}.$$

This says that an argument is structurally sufficient iff every explicit attack on it is itself neutralized by some explicit counterattack. GRASP is then introduced as a graded, propagation-based analogue of this intuition: instead of binary sufficient/insufficient labels, it yields continuous scores.

The associated principles are explicit. Attack sensitivity says unneutralized attacks defeat sufficiency. Defense reinstatement says attacking an attacker restores sufficiency with respect to that attack. Structural locality says only structurally connected arguments matter. Baseline sufficiency says unattacked arguments are sufficient by default. These are the semantic commitments that the continuous scoring rule is meant to preserve.

Just as important are the paper’s negative claims. GRASP does not measure truth, factual correctness, ethical correctness, persuasiveness, rhetorical appeal, style, verbosity, or generic “convincingness.” A high GRASP score therefore licenses only a narrow inference: the argument is robustly placed in the explicit attack–defense graph under the chosen graph-construction procedure.

2. Graph formalization and propagation operator

GRASP operates on an explicit interaction graph. The abstract structural form is

$\mathcal{G}=(A,R^-,R^+),$

where $A$ is a finite set of arguments, $R^- \subseteq A\times A$ is attack, and $R^+ \subseteq A\times A$ is support. The notation $(b,a)\in R^-$ means that $b$ attacks $a$, and $(c,a)\in R^+$ means that $c$ supports $\mathcal{G}=(A,R^-,R^+),$0.

For the actual ranking operator, the paper uses a weighted version,

$\mathcal{G}=(A,R^-,R^+),$1

with attack matrix $\mathcal{G}=(A,R^-,R^+),$2 and defense matrix $\mathcal{G}=(A,R^-,R^+),$3, where $\mathcal{G}=(A,R^-,R^+),$4. Here $\mathcal{G}=(A,R^-,R^+),$5 is the strength with which $\mathcal{G}=(A,R^-,R^+),$6 attacks $\mathcal{G}=(A,R^-,R^+),$7, and $\mathcal{G}=(A,R^-,R^+),$8 is the extent to which $\mathcal{G}=(A,R^-,R^+),$9 defends $A$0. In the main experiments, defense is not judged directly. Instead,

$A$1

so that a two-hop path $A$2 is treated as defense of $A$3 by $A$4, since $A$5 attacks an attacker of $A$6.

The scoring object is a nonnegative strength vector $A$7. The core GRASP operator is

$A$8

with $A$9 as attack and defense influence parameters. The denominator aggregates total incoming attack weighted by attacker strength, while the numerator aggregates total incoming defense weighted by defender strength. The additive constants $R^- \subseteq A\times A$0 provide a neutral baseline, so unattacked and undefended arguments begin from a baseline strength rather than zero. Relative to older attack-only ranking schemes such as the H-categorizer, the key move is explicit defense in the numerator.

A damped version is then introduced:

$R^- \subseteq A\times A$1

and the update rule is

$R^- \subseteq A\times A$2

with uniform initialization $R^- \subseteq A\times A$3. In the main experiments, unless otherwise stated, the hyperparameters are fixed a priori to

$R^- \subseteq A\times A$4

The stopping rule is

$R^- \subseteq A\times A$5

with tol=1e-10 and max_iters=2000, and the final ranking is the descending sort of the converged $R^- \subseteq A\times A$6.

In matrix form, the same operator is written as

$R^- \subseteq A\times A$7

so $R^- \subseteq A\times A$8 is total incoming weighted attack and $R^- \subseteq A\times A$9 is total incoming weighted defense. This is a propagation operator because node scores depend on adjacent attackers and defenders, and through iteration also on higher-order graph effects (Misra et al., 18 May 2026).

3. Convergence theory and relation to ranking semantics

The paper provides a fixed-point analysis for the nonlinear operator. Its main result is stated on

$R^+ \subseteq A\times A$0

If $R^+ \subseteq A\times A$1 have non-negative entries and

$R^+ \subseteq A\times A$2

with $R^+ \subseteq A\times A$3, then $R^+ \subseteq A\times A$4 and $R^+ \subseteq A\times A$5 is a contraction on $R^+ \subseteq A\times A$6. Consequently, $R^+ \subseteq A\times A$7 admits a unique fixed point $R^+ \subseteq A\times A$8, and the iteration $R^+ \subseteq A\times A$9 converges to $(b,a)\in R^-$0. The result extends to the damped variant as well.

The proof strategy is standard fixed-point analysis applied to a nonstandard operator. The paper first establishes invariance of the set $(b,a)\in R^-$1, then derives a Lipschitz bound. It is explicit that the bounds on $(b,a)\in R^-$2 and $(b,a)\in R^-$3 are worst-case sufficient rather than necessary, and that in practice these parameters are treated as tunable hyperparameters.

The discussion of damping is also precise. Damping can improve numerical stability in practice, but it does not automatically turn a non-contraction into a contraction. The appendix writes

$(b,a)\in R^-$4

and proves

$(b,a)\in R^-$5

so if the base operator has Lipschitz constant $(b,a)\in R^-$6, damping alone does not force it below $(b,a)\in R^-$7. This is a theorem about contraction analysis, not a claim that damping is computationally useless.

Within argumentation theory, GRASP is positioned as a new member of the gradual/ranking-based family. Like the H-categorizer and related ranking semantics, it assigns continuous strengths rather than extension-based acceptability sets. Unlike simple attack-only methods, it includes explicit defense through the numerator. The paper’s novelty claim is therefore not that graph propagation itself is new, but that it proposes a specific damped attack–defense propagation operator with a convergence guarantee and uses it as a transparency-preserving aggregation layer over LLM-induced interaction graphs (Misra et al., 18 May 2026).

4. End-to-end pipeline and benchmark construction

The operational pipeline begins with raw debate data. In the paper’s main benchmark, StructDebate, each debate contains a set of arguments. For every ordered pair $(b,a)\in R^-$8, a judge LLM is prompted with a directed local task: score how strongly $(b,a)\in R^-$9 attacks or undermines $b$0. The prompt defines an attack_score in $b$1, where $b$2 means unrelated or supportive and $b$3 means direct contradiction or strong refutation. These pairwise scores populate the dense attack matrix $b$4. GRASP then constructs $b$5, by default via $b$6, initializes $b$7, iterates the damped propagation operator until convergence, and ranks arguments by descending converged score.

The paper explicitly identifies the computational cost of this design. For dense $b$8, implementation is $b$9 per iteration, and constructing $a$0 also requires $a$1 pairwise LLM judgments. This $a$2 pairwise interaction scoring cost is listed as a limitation.

StructDebate contains 7,000 machine-generated arguments across 300 debates, drawn from 50 real motions and two generation regimes: a pool setting with independently generated arguments and a multi-turn setting with 10-turn self-debates. Six judge models independently induce attack matrices $a$3, and rankings are compared with Kendall’s $a$4, Spearman’s $a$5, normalized Kendall swap distance, and Top-$a$6 overlap.

The comparison baselines are deliberately structured. RAW asks an LLM to rank arguments directly from the whole debate text. RAW+SS gives the judge an additional verbal definition of structural sufficiency before asking for a ranking. This tests whether holistic instability is merely a matter of underspecified instructions. The paper’s answer is negative: clarifying the criterion verbally does not reliably solve the instability of holistic judging.

The paper also includes an external sanity check using human-written iDebate/IDEA point–counterpoint pages. There, human-authored COUNTERPOINT $a$7 POINT relations are treated as labeled attacks, and the induced attack scores are evaluated against them. This auxiliary study supports the claim that LLMs are more reliable as local interaction scorers than as holistic judges (Misra et al., 18 May 2026).

5. Empirical findings

The main empirical result is that decomposing evaluation into local judgments plus deterministic aggregation substantially improves reproducibility across judge models. In Table 1, RAW achieves Kendall $a$8 of $a$9 in Pool and $(c,a)\in R^+$0 in Multi-turn, while default GRASP reaches

$(c,a)\in R^+$1

This corresponds to improvements of $(c,a)\in R^+$2 and $(c,a)\in R^+$3, respectively. Spearman $(c,a)\in R^+$4 also rises from roughly $(c,a)\in R^+$5 to about $(c,a)\in R^+$6, and swap distance drops substantially. RAW+SS does not fix the problem: it improves Pool $(c,a)\in R^+$7 only to $(c,a)\in R^+$8 and collapses in Multi-turn to $(c,a)\in R^+$9, with worse top-$c$0 overlap.

The consensus-divergence analysis reinforces the same point. Using Borda and a greedy Kemeny approximation to build consensus rankings, the authors find that under GRASP the most divergent judge is much closer to consensus than under RAW. Their summary is that the most divergent GRASP judge is roughly $c$1 closer to consensus than the most divergent RAW judge.

A key explanatory finding is that the local attack matrices themselves are much more similar across models than the final holistic rankings are. Pairwise Pearson correlations between judge-induced attack matrices $c$2 are mostly between $c$3 and $c$4. The external sanity check on iDebate/IDEA yields mean score $c$5 for explicit COUNTERPOINT $c$6 POINT edges versus $c$7 for other incoming edges, with AUC $c$8 and Hit@5 $c$9. The paper interprets this as evidence that LLMs can be useful as local interaction scorers even when they are unstable as holistic judges.

The paper also evaluates GRASP as a ranking semantics in a synthetic structural testbed. Compared against baselines such as the H-categorizer, KatzAttack, Defense Ratio, Binary Indegree, and Max Incoming Attack, default GRASP with $\mathcal{G}=(A,R^-,R^+),$00 achieves the lowest violation rate and severity:

$\mathcal{G}=(A,R^-,R^+),$01

Attack-only GRASP, with $\mathcal{G}=(A,R^-,R^+),$02, performs worse, which supports the importance of explicit defense propagation. Alternative defense constructions such as $\mathcal{G}=(A,R^-,R^+),$03, $\mathcal{G}=(A,R^-,R^+),$04, or $\mathcal{G}=(A,R^-,R^+),$05 also perform worse than the default two-hop choice.

The case studies emphasize that GRASP is a coupled nonlinear propagation process rather than a one-step heuristic. The paper shows non-monotonic rank dynamics over iterations, including a four-node toy example in which a rank reversal occurs between iterations 1 and 2 because defending a defender indirectly changes the standing of a remote node. This illustrates the intended higher-order semantics of the propagation rule (Misra et al., 18 May 2026).

6. Interpretation, limitations, and disambiguation

A particularly important result is negative. On the Debate Decision Outcomes dataset, where the target is human convincingness or point/status wins, all structural methods perform at chance. For GRASP, convincingness and status accuracies are around $\mathcal{G}=(A,R^-,R^+),$06–$\mathcal{G}=(A,R^-,R^+),$07, and the Spearman correlation between GRASP side-score difference and convincingness margin is

$\mathcal{G}=(A,R^-,R^+),$08

The paper treats this not as failure of the method’s intended purpose, but as validation of conceptual separation: structural robustness and human convincingness are different targets.

This implies a narrow but clear use profile. GRASP should be used when one wants a graph-relative audit of argumentative robustness—for example in debate analysis, policy deliberation, legal reasoning support, or multi-agent systems where it matters which arguments are structurally well-defended in the explicit discourse. It should not be used as a standalone oracle of truth, factual validity, normative desirability, or human persuasiveness. The paper’s impact statement says it should be paired with factual verification, domain expertise, human review, and auditing of the underlying graph. If the local edge scores are biased or low-quality, the final rankings inherit those problems even though the aggregation itself is transparent.

A plausible implication is that GRASP’s main contribution is sociotechnical as much as algorithmic. It narrows the evaluative target, exposes the aggregation mechanism, and makes it possible to inspect the path from local judgments to global ranking. That is a different form of legitimacy from holistic LLM judging, not a broader claim to epistemic or normative authority.

The acronym “GRASP” is also used for unrelated work in other areas, including a grid-based benchmark for commonsense spatial reasoning (Tang et al., 2024), a benchmark for language grounding and situated physics in multimodal models (Jassim et al., 2023), a reinforcement-learning framework for geospatial segmentation (Jiang et al., 23 Aug 2025), a multi-hop QA system based on graph agentic search over propositions (Jenkins et al., 15 May 2026), and a social-reasoning dataset grounded in non-verbal interaction (Kim et al., 15 May 2026). In the context of argumentation and LLM evaluation, however, GRASP refers specifically to Gradual Ranking with Attacks and Support Propagation (Misra et al., 18 May 2026).