HerO 2 Fact-Verification Pipeline

• HerO 2 is a modular, open-source fact-verification pipeline designed for sub-1-minute per-claim inference using document-level abstractive summarization and answer-form reformulation.
• It leverages quantized large models, such as Qwen3 32B AWQ, to achieve high veracity accuracy (0.692) while operating efficiently on commodity GPUs.
• The end-to-end design incorporates offline indexing, batched summarization, and prompt caching to ensure both reproducibility and speed in shared-task evaluations.

HerO 2 is Team HUMANE’s second-generation, open-source fact-verification pipeline for the AVeriTeC 2025 shared task. It extends the original HerO system and is designed end-to-end for sub-1-minute per-claim inference on commodity GPUs such as an NVIDIA A10G with 23 GB VRAM, while targeting high Ev2R recall and veracity accuracy. Its defining modifications are document-level abstractive summarization, answer-form reformulation, 4-bit post-training quantization of the veracity backbone, and updated language-model backbones across the pipeline. In the final FEVER-25 evaluation, it placed second with an AVeriTeC score of $0.271 \pm 0.004$ and a runtime of 29.19 s per claim (Yoon et al., 15 Jul 2025).

1. Identity and nomenclature

In current arXiv usage, “HerO 2” denotes the AVeriTeC 2025 fact-verification system introduced by Team HUMANE (Yoon et al., 15 Jul 2025). The designation is easily conflated with similarly named but unrelated systems. “HeRo 2.0” refers to a low-cost differential-drive mobile robot for swarm robotics research and education (Rezeck et al., 2022), whereas “HER2” in computational pathology denotes the human epidermal growth factor receptor 2 and appears in work on augmented-reality microscope-based HER2 scoring (Zhang et al., 2020). These are distinct topics, architectures, and application domains.

Within fact verification, HerO 2 is positioned as an efficiency-oriented, shared-task system. The paper characterizes it as an enhanced version of HerO, the best-performing open-source model from the previous year’s challenge, with changes aimed at improving evidence quality, optimizing veracity prediction under computational constraints, and updating the LM backbone stack (Yoon et al., 15 Jul 2025).

2. End-to-end pipeline

HerO 2 is organized as a four-stage pipeline. The first stage, “knowledge store construction,” indexes all provided web documents with dense embeddings using gte-base-en-v1.5 and simultaneously generates paragraph-level candidates through abstractive summarization with Qwen3 8B. The second stage performs evidence retrieval: HyDE-FC with Llama3.1 8B expands a claim into a pseudo-article, dense retrieval selects the top-10 full documents, and each selected document is then re-summarized into a single paragraph. The third stage carries out question generation and answer reformulation with Qwen3 8B. The fourth stage performs veracity prediction with a fine-tuned Qwen3 32B model quantized to 4-bit with AWQ, using the claim plus the top-10 question–answer pairs as in-context examples under an “annotator rationale” prompt (Yoon et al., 15 Jul 2025).

Stage	Main model/component	Function
Knowledge store construction	gte-base-en-v1.5; Qwen3 8B	Dense indexing and paragraph-level abstractive summaries
Evidence retrieval	HyDE-FC with Llama3.1 8B	Claim expansion and top-10 document selection
Question generation and answer reformulation	Qwen3 8B	Claim-focused questions and answer-form evidence conversion
Veracity prediction	Qwen3 32B AWQ	Final label prediction from claim and top-10 Q–A pairs

This organization reflects a deliberate shift away from purely extractive retrieval. A central design choice is that evidence is repeatedly normalized into paragraph-like or answer-form representations before final judgment. This suggests an attempt to reduce mismatch between retrieved evidence granularity and the generative veracity model’s input requirements.

3. Evidence construction: summarization and reformulation

A major technical claim of HerO 2 is that document-level abstractive summarization outperforms smaller evidence units for retrieval. Rather than using extractive sentence scoring, each document $d$ is summarized by Qwen3 8B with the prompt “Summarize this document into a single, coherent paragraph capturing its main ideas…,” producing summary $s$ according to

$s^* = \arg\max_{s} P_{\theta}(s \mid d),$

with $\theta$ fixed from instruction-tuning and no additional supervised loss or parameter tuning at this stage (Yoon et al., 15 Jul 2025).

Empirically, the paper reports that paragraph-level evidence outperforms both sentence- and chunk-level retrieval. On the development set, document-level top-10 retrieval yields Ev2R recall 0.522, compared with 0.413 for 3-sentence chunks and 0.374 for sentence top-10 (Yoon et al., 15 Jul 2025). This is one of the paper’s clearest architectural conclusions: the system benefits when full documents are first compressed into coherent single-paragraph surrogates.

The answer reformulation step applies another transformation. After question generation, Qwen3 8B receives the triplet $(\text{Claim}, \text{Question}, \text{Evidence})$ and generates an answer-form string:

$$\hat{a} = \arg\max_{a} P_{\theta}(a \mid \text{Claim}, \text{Question}, \text{Evidence}),$$

again with no extra rescoring (Yoon et al., 15 Jul 2025). According to the reported results, adding summarization and reformulation increases Q+A Ev2R recall from 0.522 to 0.556 at top-10. The paper treats this as evidence that answer-shaped conditioning improves downstream evidence usefulness, not merely retrieval ranking.

4. Veracity prediction and quantized inference

The final veracity module is a fine-tuned Qwen3 32B model quantized to 4-bit using AWQ in order to satisfy the A10G’s 23 GB VRAM limit. The quantization is described at the weight-group level: if $W$ is a weight matrix, then for each group

$$s = \frac{\max(W)-\min(W)}{2^4-1}, \qquad \hat W = \mathrm{round}\!\Bigl(\frac{W}{s}\Bigr)\times s.$$

Activations remain in 16-bit to preserve generation fidelity (Yoon et al., 15 Jul 2025).

The paper reports that this reduces model size by 75% with minimal performance loss, and the quantized Qwen3 32B backbone achieves a veracity accuracy of 0.692 on development data, compared with 0.674 for the unquantized Qwen3 8B baseline and 0.588 for Llama3.1 8B. The corresponding veracity F1 for Qwen3 32B AWQ is 0.382 (Yoon et al., 15 Jul 2025). In the system description, this is not presented merely as a deployment optimization; it is integral to enabling a larger backbone under fixed hardware constraints.

The backbone choices across the pipeline are explicitly specified. HyDE-FC uses Llama3.1 8B with max tokens $= 512$, temperature $d$0, and top-$d$1. Document summarization, question generation, and answer reformulation use Qwen3 8B with temperature $d$2, top-$d$3, top-$d$4, and min-$d$5. Veracity prediction uses Qwen3 32B AWQ with temperature $d$6, top-$d$7, top-$d$8, with fallback to top-2 sampling if no label is emitted. The Qwen3 32B model was fine-tuned on FEVER-25’s train split with Adam, learning rate $d$9, batch size 128, for 2 epochs (Yoon et al., 15 Jul 2025).

5. Evaluation protocol and empirical results

HerO 2 is evaluated on the FEVER-25 train, development, and test sets. The reported metrics are Ev2R recall (Q+A) on the development set, veracity F1 and accuracy on development, AVeriTeC score defined as veracity accuracy when Q+A $s$0, and average runtime per claim on the test set (Yoon et al., 15 Jul 2025).

The development results isolate several system-level effects. Best retrieval with document-level top-10 gives Ev2R $s$1. Adding summarization and reformulation raises this to 0.556 at top-10. For veracity prediction, Qwen3 32B AWQ achieves F1 $s$2 and accuracy $s$3, outperforming the cited 8B baselines on accuracy (Yoon et al., 15 Jul 2025). These numbers tie the system’s empirical gains to two concrete mechanisms: evidence normalization and larger-but-quantized veracity inference.

The final shared-task results emphasize both ranking and latency.

System	AVeriTeC	Runtime
CTU AIC	0.332	53.7 s/claim
HerO 2	$s$4	29.19 s/claim
yellow_flash	$s$5	31.71 s/claim

HerO 2 ranked second on the leaderboard while achieving the shortest runtime among the top three systems (Yoon et al., 15 Jul 2025). The paper also states that the end-to-end pipeline executes in under 30 s per claim on an A10G, roughly half the time of the top-ranked system while retaining strong accuracy. A plausible implication is that the system was engineered for a Pareto-efficient operating point rather than for absolute leaderboard maximization alone.

6. Efficiency, reproducibility, and significance

The runtime profile is attributed to four explicit implementation decisions: embeddings are indexed offline, summarization is batched via vLLM, the 32B veracity backbone is quantized to 4 bits, and prompt templates are cached (Yoon et al., 15 Jul 2025). Together, these measures allow the system to meet the sub-60 s requirement while maintaining a multi-stage generative architecture. This is significant because AVeriTeC-style pipelines are often bottlenecked either by retrieval recall or by the cost of repeated large-model inference; HerO 2 addresses both simultaneously through evidence compression and constrained-model deployment.

The reproducibility package is comparatively detailed. The repository provides code, fine-tuned weights, Docker and Vagrant scripts, and evaluation utilities including an Ev2R wrapper for Llama 3.3 70B. The specified software stack includes Python 3.12.9, PyTorch 2.6.0, Transformers 4.51.3, vLLM 0.8.5, and Sentence-Transformers 4.1.0, alongside step-by-step instructions for replication under the shared-task VM environment (Yoon et al., 15 Jul 2025).

HerO 2’s broader methodological importance lies in how it reframes evidence preparation. The system does not rely solely on better retrieval ranking; instead, it transforms retrieved documents into a sequence of increasingly task-aligned representations: dense-indexed documents, single-paragraph summaries, claim-focused questions, and answer-form responses. This suggests a general architecture for fact verification in which representational alignment between retrieval outputs and veracity inputs is treated as a first-class design objective.

7. Position within the literature

Relative to the original HerO, HerO 2 is described as an enhanced version that improves evidence quality through document summarization and answer reformulation, optimizes veracity prediction via post-training quantization under computational constraints, and integrates updated language-model backbones (Yoon et al., 15 Jul 2025). The resulting system occupies a specific niche in the literature: an open-source, shared-task-grade verifier optimized for practical latency on commodity hardware rather than an unconstrained, maximum-scale architecture.

A frequent misconception arises from the name itself. HerO 2 is unrelated to “HeRo 2.0: A Low-Cost Robot for Swarm Robotics Research,” which is a ROS-integrated mobile-robot platform composed of a 3D-printed body and open-source software (Rezeck et al., 2022). It is likewise unrelated to HER2 scoring systems in pathology, such as the augmented-reality microscope workflow for immunohistochemistry-based breast-cancer assessment (Zhang et al., 2020). In research indexing and citation, the capitalization pattern and the presence or absence of the decimal notation “2.0” matter.

Within fact verification proper, HerO 2 can be understood as a modular pipeline whose empirical contribution is not a single new model but a coordinated set of design decisions: abstractive document summarization, answer-form evidence conversion, 4-bit AWQ quantization, and updated LM backbones. The shared-task results indicate that this combination yielded a competitive balance of evidence recall, veracity accuracy, and inference speed (Yoon et al., 15 Jul 2025).