GAAR: Automated Argument Reconstruction

• GAAR is a modular LLM-centric pipeline for explicit argument reconstruction, decomposing natural language arguments into logical forms.
• It leverages LLM inference combined with symbolic reasoning and SAT-based validity checks to ensure faithful and minimal argument representations.
• Empirical evaluations on the Arguinas dataset show significant improvements in critical thinking benchmarks and reasoning tasks.

GAAR refers to the Generalized Automatic Argument Reconstruction engine, a modular large-language-model (LLM)-centric pipeline for explicit argument reconstruction from natural language. GAAR formalizes argument decomposition, logical validity checking, and faithfulness assessment, enabling the synthesis of high-quality reconstructions that demonstrably improve LLM performance on a range of critical thinking benchmarks (Ryu et al., 18 Mar 2026).

1. Architectural Structure and Operation

GAAR decomposes natural language arguments into a formal, multi-stage process that outputs: (1) explicit and implicit premises $P = \{p_1, ..., p_n\}$, (2) an explicit conclusion $c$, and (3) optional fallacy labels. The pipeline sequence comprises:

• Fallacy Detection: Flags formal/informal logical fallacies in the source text, e.g., "affirming the consequent" or "false equivalence".
• Initial Reconstruction: Leverages the detected fallacies and argument-type priors (general categories or Walton schemes) to construct an initial English argument tree $(P_0, c_0)$.
• Formalization: Maps each premise and the conclusion to explicit first-order logic (FOL) formulas, along with a symbol dictionary.
• Validity Judgment and Premise Pruning: Uses a SAT (specifically Z3)-based solver to assess whether $\bigwedge \phi(p_i) \Rightarrow \phi(c)$ and prunes to the minimal sufficient premise set.
• Streamlining (Back-translation): Reverts pruned logical forms to natural language, aligning reconstruction $(P_1, c_1)$ to the mathematical structure.
• Faithfulness Judgment: Assesses the output for accuracy, completeness, and parsimony. Fails trigger iterative refinement with task-specific LLM revision prompts.

The architecture integrates LLM inference (Claude Sonnet 4.5) with formal symbolic reasoning, iterating until strict faithfulness is achieved or a maximum iteration is reached (Ryu et al., 18 Mar 2026).

2. Formal Model and Mathematical Definitions

Given an input argument $x$, GAAR produces $r = (P, c)$ where $P$ and $c$ are in natural language, and $\phi: \text{NL} \to \text{FOL}$ formalizes these into logic. Logical validity requires

$c$0

Minimal premise sets are computed such that all $c$1 with $c$2 are found, retaining only the premises present in at least one minimal proof. Probabilistically, if $c$3 is the LLM model score, the highest-probability reconstruction is: $c$4 For supervised learning (as in Arguinas pretraining) the token-level cross-entropy is used: $c$5 Validity is verified by checking the unsatisfiability of $c$6. Only premises necessary for deduction are retained (Ryu et al., 18 Mar 2026).

3. Algorithmic Implementation and Core Procedures

GAAR operates as an iterative loop over the following steps:

$(P_0, c_0)$3

Each subroutine is executed via an LLM prompt, except for the Z3-based SAT solver in ValidityAndPrune, which is implemented as an LLM-generated Python script (Ryu et al., 18 Mar 2026).

4. Arguinas Dataset and Empirical Evaluation

As an oracle, GAAR synthesized the Arguinas dataset:

Attribute	Value
Total argument samples	2,850
Average argument length (words)	$c$7
Average premises per reconstruction	$c$8
Implicit premise rate	$c$9

Seven distinct sources were used: historical handbooks, ProCon.org, NYT debates, synthetic GPT-5 arguments, and fallacious LLM-injected examples. Claude Sonnet 4.5 was selected from 13 LLMs via automatic tournament evaluation (TOPSIS). Faithfulness and NL$(P_0, c_0)$0FOL translation were confirmed by human and automated assessments (99.0% FOL translation accuracy, 89.5% faithfulness agreement, Cohen's $(P_0, c_0)$1) (Ryu et al., 18 Mar 2026).

5. Impact on Critical Thinking Benchmarks

The downstream impact was measured on seven established tasks, including argument quality evaluation (WebisArgQuality20, UKPConvArg2), argument reasoning (ArgsNovel, ArgRC), legal reasoning (LegalArg), and logical reasoning (ReClor):

• Pre-adaptive finetuning (Arguinas SFT $(P_0, c_0)$2 downstream task SFT): On Qwen3-4B and 8B, Arguinas pre-adaption outperformed direct finetuning and other baselines on 6/7 tasks, achieving the largest gains for ArgRC (+3.3% to +5.3%) and LegalArg (+10% to +12%). With as little as 10% downstream data, the Arguinas-adapted model matched full-data baselines.
• Continued finetuning (Arguinas SFT only): For Qwen2.5-7B-Instruct, up to +51.3% Macro F1 gains were observed on WebisArgQuality20 without direct SFT for downstream tasks. Gains were confirmed across all evaluation sets (Ryu et al., 18 Mar 2026).

6. Strengths, Limitations, and Future Directions

GAAR’s architecture provides generality (coverage of arbitrary domains, argument types, all major Walton schemes), robust symbolic integration (SAT-based validity, minimal premises), and strong data efficiency. The faithfulness criteria of accuracy, completeness, and parsimony enforce reconstruction quality, exceeding prior engines.

Limitations include high computation and cost (multiple LLM passes, solver calls), dependence on LLM reliability, and lack of end-to-end differentiability. Highly ambiguous or rhetorical input can still generate suboptimal reconstructions.

Future extensions proposed by the original authors include distilling the pipeline into a single finetuned LLM or sparse-mixture model, supporting interactive or human-in-the-loop workflows, handling dialectic structures and multi-modal arguments, and embedding reconstructed structures as reasoning priors for LLM architectures (Ryu et al., 18 Mar 2026).

---

GAAR represents a scalable, hybrid LLM-symbolic approach for explicit, faithful argument structure extraction. Its formal integration of natural language processing with logical reasoning and faithfulness assessment furnishes both a powerful dataset (Arguinas) and a practical engine for advancing research in LLM reasoning and argument analysis.