---
title: HerO 2 Fact-Verification Pipeline
url: https://www.emergentmind.com/topics/hero-2
type: topic
---

# HerO 2 Fact-Verification Pipeline

HerO 2 is Team HUMANE’s second-generation, open-source fact-verification pipeline for the AVeriTeC 2025 shared task. It extends the original HerO system and is designed end-to-end for sub-1-minute per-claim inference on commodity GPUs such as an NVIDIA A10G with 23 GB VRAM, while targeting high Ev2R recall and veracity accuracy. Its defining modifications are document-level abstractive summarization, answer-form reformulation, 4-bit post-training quantization of the veracity backbone, and updated language-model backbones across the pipeline. In the final FEVER-25 evaluation, it placed second with an AVeriTeC score of \(0.271 \pm 0.004\) and a runtime of 29.19 s per claim [2507.11004].

## 1. Identity and nomenclature

In current arXiv usage, “HerO 2” denotes the AVeriTeC 2025 fact-verification system introduced by Team HUMANE [2507.11004]. The designation is easily conflated with similarly named but unrelated systems. “HeRo 2.0” refers to a low-cost differential-drive mobile robot for swarm robotics research and education [2202.12391], whereas “HER2” in computational pathology denotes the human epidermal growth factor receptor 2 and appears in work on augmented-reality microscope-based HER2 scoring [2009.06816]. These are distinct topics, architectures, and application domains.

Within fact verification, HerO 2 is positioned as an efficiency-oriented, shared-task system. The paper characterizes it as an enhanced version of HerO, the best-performing open-source model from the previous year’s challenge, with changes aimed at improving evidence quality, optimizing veracity prediction under computational constraints, and updating the LM backbone stack [2507.11004].

## 2. End-to-end pipeline

HerO 2 is organized as a four-stage pipeline. The first stage, “knowledge store construction,” indexes all provided web documents with dense embeddings using gte-base-en-v1.5 and simultaneously generates paragraph-level candidates through abstractive summarization with Qwen3 8B. The second stage performs evidence retrieval: HyDE-FC with Llama3.1 8B expands a claim into a pseudo-article, dense retrieval selects the top-10 full documents, and each selected document is then re-summarized into a single paragraph. The third stage carries out question generation and answer reformulation with Qwen3 8B. The fourth stage performs veracity prediction with a fine-tuned Qwen3 32B model quantized to 4-bit with AWQ, using the claim plus the top-10 question–answer pairs as in-context examples under an “annotator rationale” prompt [2507.11004].

| Stage | Main model/component | Function |
|---|---|---|
| Knowledge store construction | gte-base-en-v1.5; Qwen3 8B | Dense indexing and paragraph-level abstractive summaries |
| Evidence retrieval | HyDE-FC with Llama3.1 8B | Claim expansion and top-10 document selection |
| Question generation and answer reformulation | Qwen3 8B | Claim-focused questions and answer-form evidence conversion |
| Veracity prediction | Qwen3 32B AWQ | Final label prediction from claim and top-10 Q–A pairs |

This organization reflects a deliberate shift away from purely extractive retrieval. A central design choice is that evidence is repeatedly normalized into paragraph-like or answer-form representations before final judgment. This suggests an attempt to reduce mismatch between retrieved evidence granularity and the generative veracity model’s input requirements.

## 3. Evidence construction: summarization and reformulation

A major technical claim of HerO 2 is that document-level abstractive summarization outperforms smaller evidence units for retrieval. Rather than using extractive sentence scoring, each document \(d\) is summarized by Qwen3 8B with the prompt “Summarize this document into a single, coherent paragraph capturing its main ideas…,” producing summary \(s\) according to

$$
s^* = \arg\max_{s} P_{\theta}(s \mid d),
$$

with \(\theta\) fixed from instruction-tuning and no additional supervised loss or parameter tuning at this stage [2507.11004].

Empirically, the paper reports that paragraph-level evidence outperforms both sentence- and chunk-level retrieval. On the development set, document-level top-10 retrieval yields Ev2R recall 0.522, compared with 0.413 for 3-sentence chunks and 0.374 for sentence top-10 [2507.11004]. This is one of the paper’s clearest architectural conclusions: the system benefits when full documents are first compressed into coherent single-paragraph surrogates.

The answer reformulation step applies another transformation. After question generation, Qwen3 8B receives the triplet \((\text{Claim}, \text{Question}, \text{Evidence})\) and generates an answer-form string:

$$
\hat{a} = \arg\max_{a} P_{\theta}(a \mid \text{Claim}, \text{Question}, \text{Evidence}),
$$

again with no extra rescoring [2507.11004]. According to the reported results, adding summarization and reformulation increases Q+A Ev2R recall from 0.522 to 0.556 at top-10. The paper treats this as evidence that answer-shaped conditioning improves downstream evidence usefulness, not merely retrieval ranking.

## 4. Veracity prediction and quantized inference

The final veracity module is a fine-tuned Qwen3 32B model quantized to 4-bit using AWQ in order to satisfy the A10G’s 23 GB VRAM limit. The quantization is described at the weight-group level: if \(W\) is a weight matrix, then for each group

$$
s = \frac{\max(W)-\min(W)}{2^4-1}, \qquad
\hat W = \mathrm{round}\!\Bigl(\frac{W}{s}\Bigr)\times s.
$$

Activations remain in 16-bit to preserve generation fidelity [2507.11004].

The paper reports that this reduces model size by 75% with minimal performance loss, and the quantized Qwen3 32B backbone achieves a veracity accuracy of 0.692 on development data, compared with 0.674 for the unquantized Qwen3 8B baseline and 0.588 for Llama3.1 8B. The corresponding veracity F1 for Qwen3 32B AWQ is 0.382 [2507.11004]. In the system description, this is not presented merely as a deployment optimization; it is integral to enabling a larger backbone under fixed hardware constraints.

The backbone choices across the pipeline are explicitly specified. HyDE-FC uses Llama3.1 8B with max tokens \(= 512\), temperature \(= 0.7\), and top-\(p = 1.0\). Document summarization, question generation, and answer reformulation use Qwen3 8B with temperature \(= 0.7\), top-\(p = 0.8\), top-\(k = 20\), and min-\(p = 0\). Veracity prediction uses Qwen3 32B AWQ with temperature \(= 0.9\), top-\(p = 0.7\), top-\(k = 1\), with fallback to top-2 sampling if no label is emitted. The Qwen3 32B model was fine-tuned on FEVER-25’s train split with Adam, learning rate \(2 \times 10^{-5}\), batch size 128, for 2 epochs [2507.11004].

## 5. Evaluation protocol and empirical results

HerO 2 is evaluated on the FEVER-25 train, development, and test sets. The reported metrics are Ev2R recall (Q+A) on the development set, veracity F1 and accuracy on development, AVeriTeC score defined as veracity accuracy when Q+A \(> 0.5\), and average runtime per claim on the test set [2507.11004].

The development results isolate several system-level effects. Best retrieval with document-level top-10 gives Ev2R \(= 0.522\). Adding summarization and reformulation raises this to 0.556 at top-10. For veracity prediction, Qwen3 32B AWQ achieves F1 \(= 0.382\) and accuracy \(= 0.692\), outperforming the cited 8B baselines on accuracy [2507.11004]. These numbers tie the system’s empirical gains to two concrete mechanisms: evidence normalization and larger-but-quantized veracity inference.

The final shared-task results emphasize both ranking and latency.

| System | AVeriTeC | Runtime |
|---|---:|---:|
| CTU AIC | 0.332 | 53.7 s/claim |
| HerO 2 | \(0.271 \pm 0.004\) | 29.19 s/claim |
| yellow_flash | \(0.253 \pm 0.005\) | 31.71 s/claim |

HerO 2 ranked second on the leaderboard while achieving the shortest runtime among the top three systems [2507.11004]. The paper also states that the end-to-end pipeline executes in under 30 s per claim on an A10G, roughly half the time of the top-ranked system while retaining strong accuracy. A plausible implication is that the system was engineered for a Pareto-efficient operating point rather than for absolute leaderboard maximization alone.

## 6. Efficiency, reproducibility, and significance

The runtime profile is attributed to four explicit implementation decisions: embeddings are indexed offline, summarization is batched via vLLM, the 32B veracity backbone is quantized to 4 bits, and prompt templates are cached [2507.11004]. Together, these measures allow the system to meet the sub-60 s requirement while maintaining a multi-stage generative architecture. This is significant because AVeriTeC-style pipelines are often bottlenecked either by retrieval recall or by the cost of repeated large-model inference; HerO 2 addresses both simultaneously through evidence compression and constrained-model deployment.

The reproducibility package is comparatively detailed. The repository provides code, fine-tuned weights, Docker and Vagrant scripts, and evaluation utilities including an Ev2R wrapper for Llama 3.3 70B. The specified software stack includes Python 3.12.9, PyTorch 2.6.0, Transformers 4.51.3, vLLM 0.8.5, and Sentence-Transformers 4.1.0, alongside step-by-step instructions for replication under the shared-task VM environment [2507.11004].

HerO 2’s broader methodological importance lies in how it reframes evidence preparation. The system does not rely solely on better retrieval ranking; instead, it transforms retrieved documents into a sequence of increasingly task-aligned representations: dense-indexed documents, single-paragraph summaries, claim-focused questions, and answer-form responses. This suggests a general architecture for fact verification in which representational alignment between retrieval outputs and veracity inputs is treated as a first-class design objective.

## 7. Position within the literature

Relative to the original HerO, HerO 2 is described as an enhanced version that improves evidence quality through document summarization and answer reformulation, optimizes veracity prediction via post-training quantization under computational constraints, and integrates updated language-model backbones [2507.11004]. The resulting system occupies a specific niche in the literature: an open-source, shared-task-grade verifier optimized for practical latency on commodity hardware rather than an unconstrained, maximum-scale architecture.

A frequent misconception arises from the name itself. HerO 2 is unrelated to “HeRo 2.0: A Low-Cost Robot for Swarm Robotics Research,” which is a ROS-integrated mobile-robot platform composed of a 3D-printed body and open-source software [2202.12391]. It is likewise unrelated to HER2 scoring systems in pathology, such as the augmented-reality microscope workflow for immunohistochemistry-based breast-cancer assessment [2009.06816]. In research indexing and citation, the capitalization pattern and the presence or absence of the decimal notation “2.0” matter.

Within fact verification proper, HerO 2 can be understood as a modular pipeline whose empirical contribution is not a single new model but a coordinated set of design decisions: abstractive document summarization, answer-form evidence conversion, 4-bit AWQ quantization, and updated LM backbones. The shared-task results indicate that this combination yielded a competitive balance of evidence recall, veracity accuracy, and inference speed [2507.11004].

Source: https://www.emergentmind.com/topics/hero-2